agora inbox for [email protected]
help / color / mirror / Atom feedRe: Protect syscache from bloating with negative cache entries
272+ messages / 6 participants
[nested] [flat]
* Re: Protect syscache from bloating with negative cache entries
@ 2020-01-14 03:49 Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
0 siblings, 1 reply; 272+ messages in thread
From: Kyotaro Horiguchi @ 2020-01-14 03:49 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
This is a new complete workable patch after a long time of struggling
with benchmarking.
At Tue, 19 Nov 2019 19:48:10 +0900 (JST), Kyotaro Horiguchi <[email protected]> wrote in
> I ran the run2.sh script attached, which runs catcachebench2(), which
> asks SearchSysCache3() for cached entries (almost) 240000 times per
> run. The number of each output line is the mean of 3 times runs, and
> stddev. Lines are in "time" order and edited to fit here. "gen_tbl.pl
> | psql" creates a database for the benchmark. catcachebench2() runs
> the shortest path in the three in the attached benchmark program.
>
> (pg_ctl start)
> $ perl gen_tbl.pl | psql ...
> (pg_ctl stop)
I wonder why I took the average of the time instead of choose the
fastest one. This benchmark is extremely CPU intensive so the fastest
run reliably represents the perfromance.
I changed the benchmark so that it shows the time of the fastest run
(run4.sh). Based on the latest result, I used the pattern 3
(SearchSyscacheN indirection, but wrongly pointed as 1 in the last
mail) in the latest version,
I took the number of the fastest time among 3 iteration of 5 runs of
both master/patched O2 binaries.
version | min
---------+---------
master | 7986.65
patched | 7984.47 = 'indirect' below
I would say this version doesn't get degradaded by indirect calls.
So, I applied the other part of the catcache expiration patch as the
succeeding parts. After that I got somewhat strange but very stable
result. Just adding struct members acceleartes the benchmark. The
numbers are the fastest time of 20 runs of the bencmark in 10 times
iterations.
ms
master 7980.79 # the master with the benchmark extension (0001)
=====
base 7340.96 # add only struct members and a GUC variable. (0002)
indirect 7998.68 # call SearchCatCacheN indirectly (0003)
=====
expire-off 7422.30 # CatCache expiration (0004)
# (catalog_cache_prune_min_age = -1)
expire-on 7861.13 # CatCache expiration (catalog_cache_prune_min_age = 0)
The patch accelerates CatCaCheSearch for uncertain reasons. I'm not
sure what makes the difference between about 8000ms and about 7400 ms,
though. Several times building of all versions then running the
benchmark gave me the results with the same tendency. I once stop this
work at this point and continue later. The following files are
attached.
0001-catcache-benchmark-extension.patch:
benchmnark extension used by the benchmarking here. The test tables
are generated using gentbl2.pl attached. (perl gentbk2.pl | psql)
0002-base_change.patch:
Preliminary adds some struct members and a GUC variable to see if
they cause any extent of degradation.
0003-Make-CatCacheSearchN-indirect-functions.patch:
Rewrite to change CatCacheSearchN functions to be called indirectly.
0004-CatCache-expiration-feature.patch:
Add CatCache expiration feature.
gentbl2.pl: A script that emits SQL statements to generate test tables.
run4.sh : The test script I used for benchmarkiing here.
build2.sh : A script I used to build the four types of binaries used here.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
@ 2020-01-21 16:29 ` Tomas Vondra <[email protected]>
2020-01-21 18:45 ` Re: Protect syscache from bloating with negative cache entries Alvaro Herrera <[email protected]>
2020-01-22 05:38 ` Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
0 siblings, 2 replies; 272+ messages in thread
From: Tomas Vondra @ 2020-01-21 16:29 UTC (permalink / raw)
To: Kyotaro Horiguchi <[email protected]>; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Hello Kyotaro-san,
I see this patch is stuck in WoA since 2019/12/01, although there's a
new patch version from 2020/01/14. But the patch seems to no longer
apply, at least according to https://commitfest.cputube.org :-( So at
this point the status is actually correct.
Not sure about the appveyor build (it seems to be about jsonb_set_lax),
but on travis it fails like this:
catcache.c:820:1: error: no previous prototype for ‘CatalogCacheFlushCatalog2’ [-Werror=missing-prototypes]
so I'll leave it in WoA for now.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
@ 2020-01-21 18:45 ` Alvaro Herrera <[email protected]>
2020-01-21 19:17 ` Re: Protect syscache from bloating with negative cache entries Tom Lane <[email protected]>
1 sibling, 1 reply; 272+ messages in thread
From: Alvaro Herrera @ 2020-01-21 18:45 UTC (permalink / raw)
To: Tomas Vondra <[email protected]>; +Cc: Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
On 2020-Jan-21, Tomas Vondra wrote:
> Not sure about the appveyor build (it seems to be about jsonb_set_lax),
> but on travis it fails like this:
>
> catcache.c:820:1: error: no previous prototype for ‘CatalogCacheFlushCatalog2’ [-Werror=missing-prototypes]
Hmm ... travis is running -Werror? That seems overly strict. I think
we shouldn't punt a patch because of that.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
2020-01-21 18:45 ` Re: Protect syscache from bloating with negative cache entries Alvaro Herrera <[email protected]>
@ 2020-01-21 19:17 ` Tom Lane <[email protected]>
2020-01-22 02:49 ` Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-22 05:03 ` Re: Protect syscache from bloating with negative cache entries Michael Paquier <[email protected]>
0 siblings, 2 replies; 272+ messages in thread
From: Tom Lane @ 2020-01-21 19:17 UTC (permalink / raw)
To: Alvaro Herrera <[email protected]>; +Cc: Tomas Vondra <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Alvaro Herrera <[email protected]> writes:
> On 2020-Jan-21, Tomas Vondra wrote:
>> Not sure about the appveyor build (it seems to be about jsonb_set_lax),
FWIW, I think I fixed jsonb_set_lax yesterday, so that problem should
be gone the next time the cfbot tries this.
>> but on travis it fails like this:
>> catcache.c:820:1: error: no previous prototype for ‘CatalogCacheFlushCatalog2’ [-Werror=missing-prototypes]
> Hmm ... travis is running -Werror? That seems overly strict. I think
> we shouldn't punt a patch because of that.
Why not? We're not going to allow pushing a patch that throws warnings
on common compilers. Or if that does happen, some committer is going
to have to spend time cleaning it up. Better to clean it up sooner.
(There is, btw, at least one buildfarm animal using -Werror.)
regards, tom lane
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
2020-01-21 18:45 ` Re: Protect syscache from bloating with negative cache entries Alvaro Herrera <[email protected]>
2020-01-21 19:17 ` Re: Protect syscache from bloating with negative cache entries Tom Lane <[email protected]>
@ 2020-01-22 02:49 ` Kyotaro Horiguchi <[email protected]>
1 sibling, 0 replies; 272+ messages in thread
From: Kyotaro Horiguchi @ 2020-01-22 02:49 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Hello.
At Tue, 21 Jan 2020 14:17:53 -0500, Tom Lane <[email protected]> wrote in
> Alvaro Herrera <[email protected]> writes:
> > On 2020-Jan-21, Tomas Vondra wrote:
> >> Not sure about the appveyor build (it seems to be about jsonb_set_lax),
>
> FWIW, I think I fixed jsonb_set_lax yesterday, so that problem should
> be gone the next time the cfbot tries this.
>
> >> but on travis it fails like this:
> >> catcache.c:820:1: error: no previous prototype for ‘CatalogCacheFlushCatalog2’ [-Werror=missing-prototypes]
>
> > Hmm ... travis is running -Werror? That seems overly strict. I think
> > we shouldn't punt a patch because of that.
>
> Why not? We're not going to allow pushing a patch that throws warnings
> on common compilers. Or if that does happen, some committer is going
> to have to spend time cleaning it up. Better to clean it up sooner.
>
> (There is, btw, at least one buildfarm animal using -Werror.)
Mmm. The cause of the error is tentative (or crude or brute)
benchmarking function provided as an extension which is not actually a
part of the patch and was included for reviewer's convenience.
Howerver, I don't want it work on Windows build. If that is regarded
as a reason for being punt, I'll repost a new version without the
benchmark soon.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
2020-01-21 18:45 ` Re: Protect syscache from bloating with negative cache entries Alvaro Herrera <[email protected]>
2020-01-21 19:17 ` Re: Protect syscache from bloating with negative cache entries Tom Lane <[email protected]>
@ 2020-01-22 05:03 ` Michael Paquier <[email protected]>
1 sibling, 0 replies; 272+ messages in thread
From: Michael Paquier @ 2020-01-22 05:03 UTC (permalink / raw)
To: Tom Lane <[email protected]>; +Cc: Alvaro Herrera <[email protected]>; Tomas Vondra <[email protected]>; Kyotaro Horiguchi <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
On Tue, Jan 21, 2020 at 02:17:53PM -0500, Tom Lane wrote:
> Alvaro Herrera <[email protected]> writes:
>> Hmm ... travis is running -Werror? That seems overly strict. I think
>> we shouldn't punt a patch because of that.
>
> Why not? We're not going to allow pushing a patch that throws warnings
> on common compilers. Or if that does happen, some committer is going
> to have to spend time cleaning it up. Better to clean it up sooner.
>
> (There is, btw, at least one buildfarm animal using -Werror.)
I agree that it is good to have in Mr Robot. More early detection
means less follow-up cleanup.
--
Michael
Attachments:
[application/pgp-signature] signature.asc (833B, ../../[email protected]/2-signature.asc)
download
^ permalink raw reply [nested|flat] 272+ messages in thread
* Re: Protect syscache from bloating with negative cache entries
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Re: Protect syscache from bloating with negative cache entries Tomas Vondra <[email protected]>
@ 2020-01-22 05:38 ` Kyotaro Horiguchi <[email protected]>
1 sibling, 0 replies; 272+ messages in thread
From: Kyotaro Horiguchi @ 2020-01-22 05:38 UTC (permalink / raw)
To: [email protected]; +Cc: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
At Tue, 21 Jan 2020 17:29:47 +0100, Tomas Vondra <[email protected]> wrote in
> I see this patch is stuck in WoA since 2019/12/01, although there's a
> new patch version from 2020/01/14. But the patch seems to no longer
> apply, at least according to https://commitfest.cputube.org :-( So at
> this point the status is actually correct.
>
> Not sure about the appveyor build (it seems to be about
> jsonb_set_lax),
> but on travis it fails like this:
>
> catcache.c:820:1: error: no previous prototype for
> ‘CatalogCacheFlushCatalog2’ [-Werror=missing-prototypes]
I changed my mind to attach the benchmark patch as .txt file,
expecting the checkers not picking up it as a part of the patchset.
I have in the precise performance measurement mode for a long time,
but I think it's settled. I'd like to return to normal mode and
explain this patch.
=== Motive of the patch
System cache is a mechanism that accelerates access to system catalogs
Basically the entries in a cache is removed via invalidation machanism
when corresponding system catalog entry is removed. On the other hand
the system cache also holds "negative" entries that indicates that the
object is nonexistent, which entry accelerates the response for
nonexistent objects. But the negative cache doesn't have a chance of
removal.
On a long-lived session that accepts a wide variety of queries on many
objects, system cache holds the cache entries for many objects that
are accessed once or a few times. Suppose every object is accessed
once per, say, 30 minutes, and the query doesn't needed to run in a
very short time. Such cache entries are almost useless but occupy a
large amount of memory.
=== Possible solutions.
Many caching system has expiration mechanism, which removes "useless"
entries to keep the size under a certain limit. The limit is
typically defined by memory usage or expiration time, in a hard or
soft way. Since we don't implement an detailed accouting of memory
usage by cache for the performance reasons, we can use coarse memory
accounting or expiration time. This patch uses expiration time
because it can be detemined on a rather clearer basis.
=== Pruning timing
The next point is when to prune cache entries. Apparently it's not
reasonable to do on every cache access time, since pruning takes a far
longer time than cache access.
The system cache is implemented on a hash. When there's no room for a new cache entry, it gets twice in size and rehashes all entries. If pruning made some space for the new entry, rehashing can be avoided, so this patch tries pruning just before enlarging hash table.
A system cache can be shrinked if less than a half of the size is
used, but this patch doesn't that. It is because we cannot predict if
the system cache that have just shrinked is going to enlarged just
after and I don't want get this patch that complex.
=== Performance
The pruning mechanism adds several entries to cache entry and updates
System cache is a very light-weight machinery so that inserting one
branch affects performance apparently. So in this patch, the new stuff
is isolated from existing code path using indirect call. After trials
on some call-points that can be indirect calls, I found that
SearchCatCache[1-4]() is the only point that doesn't affect
performance. (Please see upthread for details.) That configuraion
also allows future implements of system caches, such like shared
system caches.
The alternative SearchCatCache[1-4] functions get a bit slower because
it maintains access timestamp and access counter. Addition to that
pruning puts a certain amount of additional time if no entries are not
pruned off.
=== Pruning criteria
At the pruning time described above, every entry is examined agianst
the GUC variable catalog_cache_prune_min_age. The pruning mechanism
involves a clock-sweep-like mechanism where an entry lives longer if
it had accessed. Entry of which access counter is zero is pruned after
catalog_cache_prune_min_age. Otherwise an entry survives the pruning
round and its counter is decremented.
All the timestamp used by the stuff is "catcacheclock", which is
updated at every transaction start.
=== Concise test
The attached test1.pl can be used to replay the syscache-bloat caused
by negative entries. Setting $prune_age to -1, pruning is turned of
and you can see that the backend unlimitedly takes more and more
memory as time proceeds. Setting it to 10 or so, the memory size of
backend process will stops raising at certain amount.
=== The patch
The attached following are the patch. They have been separated for the
benchmarking reasons but that seems to make the patch more easy to
read so I leave it alone. I forgot its correct version through a long
time of benchmarking so I started from v1 now.
- v1-0001-base_change.patch
Adds new members to existing structs and catcacheclock-related code.
- v1-0002-Make-CatCacheSearchN-indirect-functions.patch
Changes SearchCatCacheN functions to be called by indirect calls.
- v1-0003-CatCache-expiration-feature.patch
The core code of the patch.
- catcache-benchmark-extension.patch.txt
The benchmarking extension that was used for benchmarking
upthread. Just for information.
- test1.pl
Test script to make syscache bloat.
The patchset doesn't contain documentaion for the new GUC option. I
will add it later.
regards.
--
Kyotaro Horiguchi
NTT Open Source Software Center
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
* [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation.
@ 2020-06-12 02:38 Andres Freund <[email protected]>
0 siblings, 0 replies; 272+ messages in thread
From: Andres Freund @ 2020-06-12 02:38 UTC (permalink / raw)
---
src/include/portability/instr_time.h | 68 ++++++++++++++++++++++++----
1 file changed, 60 insertions(+), 8 deletions(-)
diff --git a/src/include/portability/instr_time.h b/src/include/portability/instr_time.h
index fc058d548a8..8b2f9a2e707 100644
--- a/src/include/portability/instr_time.h
+++ b/src/include/portability/instr_time.h
@@ -83,7 +83,9 @@
#define PG_INSTR_CLOCK CLOCK_REALTIME
#endif
+/* time in baseline cpu cycles */
typedef int64 instr_time;
+
#define NS_PER_S INT64CONST(1000000000)
#define US_PER_S INT64CONST(1000000)
#define MS_PER_S INT64CONST(1000)
@@ -95,17 +97,67 @@ typedef int64 instr_time;
#define INSTR_TIME_SET_ZERO(t) ((t) = 0)
-static inline instr_time pg_clock_gettime_ns(void)
+#include <x86intrin.h>
+#include <cpuid.h>
+
+/*
+ * Return what the number of cycles needs to be multiplied with to end up with
+ * seconds.
+ *
+ * FIXME: The cold portion should probably be out-of-line. And it'd be better
+ * to not recompute this in every file that uses this. Best would probably be
+ * to require explicit initialization of cycles_to_sec, because having a
+ * branch really is unnecessary.
+ *
+ * FIXME: We should probably not unnecessarily use floating point math
+ * here. And it's likely that the numbers are small enough that we are running
+ * into floating point inaccuracies already. Probably worthwhile to be a good
+ * bit smarter.
+ *
+ * FIXME: This would need to be conditional, with a fallback to something not
+ * rdtsc based.
+ */
+static inline double __attribute__((const))
+get_cycles_to_sec(void)
{
- struct timespec tmp;
+ static double cycles_to_sec = 0;
- clock_gettime(PG_INSTR_CLOCK, &tmp);
+ /*
+ * Compute baseline cpu peformance, determines speed at which rdtsc advances
+ */
+ if (unlikely(cycles_to_sec == 0))
+ {
+ uint32 cpuinfo[4] = {0};
- return tmp.tv_sec * NS_PER_S + tmp.tv_nsec;
+ __get_cpuid(0x16, cpuinfo, cpuinfo + 1, cpuinfo + 2, cpuinfo + 3);
+ cycles_to_sec = 1 / ((double) cpuinfo[0] * 1000 * 1000);
+ }
+
+ return cycles_to_sec;
+}
+
+static inline instr_time pg_clock_gettime_ref_cycles(void)
+{
+ /*
+ * The rdtscp waits for all in-flight instructions to finish (but allows
+ * later instructions to start concurrently). That's good for some timing
+ * situations (when the time is supposed to cover all the work), but
+ * terrible for others (when sub-parts of work are measured, because then
+ * the pipeline stall due to the wait change the overall timing).
+ */
+#if 0
+ unsigned int aux;
+ int64 tsc = __rdtscp(&aux);
+
+ return tsc;
+#else
+
+ return __rdtsc();
+#endif
}
#define INSTR_TIME_SET_CURRENT(t) \
- (t) = pg_clock_gettime_ns()
+ (t) = pg_clock_gettime_ref_cycles()
#define INSTR_TIME_ADD(x,y) \
do { \
@@ -123,13 +175,13 @@ static inline instr_time pg_clock_gettime_ns(void)
} while (0)
#define INSTR_TIME_GET_DOUBLE(t) \
- ((double) (t) / NS_PER_S)
+ ((double) (t) * get_cycles_to_sec())
#define INSTR_TIME_GET_MILLISEC(t) \
- ((double) (t) / NS_PER_MS)
+ ((double) (t) * (get_cycles_to_sec() * MS_PER_S))
#define INSTR_TIME_GET_MICROSEC(t) \
- ((double) (t) / NS_PER_US)
+ ((double) (t) * (get_cycles_to_sec() * US_PER_S))
#else /* !HAVE_CLOCK_GETTIME */
--
2.25.0.114.g5b0ca878e0
--wn4ncs637ccpvbhb--
^ permalink raw reply [nested|flat] 272+ messages in thread
end of thread, other threads:[~2020-06-12 02:38 UTC | newest]
Thread overview: 272+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2020-01-14 03:49 Re: Protect syscache from bloating with negative cache entries Kyotaro Horiguchi <[email protected]>
2020-01-21 16:29 ` Tomas Vondra <[email protected]>
2020-01-21 18:45 ` Alvaro Herrera <[email protected]>
2020-01-21 19:17 ` Tom Lane <[email protected]>
2020-01-22 02:49 ` Kyotaro Horiguchi <[email protected]>
2020-01-22 05:03 ` Michael Paquier <[email protected]>
2020-01-22 05:38 ` Kyotaro Horiguchi <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
2020-06-12 02:38 [PATCH v1 2/2] WIP: Use cpu reference cycles, via rdtsc, to measure time for instrumentation. Andres Freund <[email protected]>
This inbox is served by agora; see mirroring instructions
for how to clone and mirror all data and code used for this inbox