You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 14:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Fri Jun 05, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3428003995341,10cyclictest3640485-21latency_hist22:35:013
3427992994731,14cyclictest3722412-21ntpq23:50:220
3427999994339,3cyclictest3671541-21kworker/u8:4+flush-259:023:35:012
3427996994340,1cyclictest131-1kworker/1:1H+kblockd19:27:111
3427999993934,4cyclictest3671541-21kworker/u8:4+flush-259:000:00:152
3427999993934,3cyclictest3754584-21kworker/u8:1+flush-259:000:35:182
3427996993932,5cyclictest335557-21systemd-journal20:45:001
3427999993732,3cyclictest3547381-21apt-get21:05:112
3427996993731,4cyclictest3582504-21ntpq21:35:211
3427992993631,3cyclictest3635604-21apt-get22:30:110
3428003993428,4cyclictest335557-21systemd-journal00:05:013
3427996993427,5cyclictest335557-21systemd-journal21:10:001
3428003993328,4cyclictest3759849-21gpgv00:30:013
3427999993329,3cyclictest3448586-21kworker/u8:1+flush-259:020:50:002
3427992993329,3cyclictest3433370-21apt-get19:15:100
3427999993224,5cyclictest335557-21systemd-journal20:32:272
3427996993228,3cyclictest3516161-21kworker/u8:0+flush-259:020:50:241
3427996993125,4cyclictest3708105-21/usr/sbin/munin23:40:141
3427992993125,4cyclictest3685927-21ntpq23:15:230
3427992993125,4cyclictest3473529-21kworker/u8:0+flush-259:020:10:140
3427999993025,3cyclictest3448586-21kworker/u8:1+events_unbound19:55:182
3427992993024,5cyclictest3649834-21ntpq22:40:200
3427999992924,3cyclictest3459322-21diskstats19:40:132
3428003992822,4cyclictest335557-21systemd-journal21:42:133
3427992992823,3cyclictest3614687-21latency_hist22:10:020
3427992992822,4cyclictest3474532-21cat19:55:010
3427996992722,4cyclictest3469254-21latency_hist19:50:011
3427992992723,3cyclictest335557-21systemd-journal20:35:010
3428003992622,3cyclictest3635384-21kworker/u8:2+flush-259:022:49:593
3427999992622,3cyclictest3728709-21kworker/u8:3+events_unbound00:15:212
3427999992622,3cyclictest3635384-21kworker/u8:2+flush-259:022:40:252
3427999992622,3cyclictest3619892-21kworker/u8:0+events_unbound22:15:102
3427999992619,6cyclictest3536868-21kworker/u8:3+flush-259:021:10:252
3427992992623,2cyclictest3448586-21kworker/u8:1+flush-259:021:05:000
3427992992620,4cyclictest3453739-21latency_hist19:35:010
3428003992520,4cyclictest3597935-21ntpq21:50:203
3428003992519,4cyclictest335557-21systemd-journal22:25:003
3428003992515,9cyclictest0-21swapper/300:10:003
3427999992520,3cyclictest3473529-21kworker/u8:0+flush-259:020:15:152
3427999992519,4cyclictest3521279-21awk20:40:012
3427992992521,3cyclictest3754901-21apt-get00:25:120
3428003992420,3cyclictest3692996-21kworker/u8:3+flush-259:023:30:173
3428003992418,4cyclictest3603055-21ntpq21:55:213
3427999992420,3cyclictest3671541-21kworker/u8:4+events_unbound23:10:262
3427999992420,3cyclictest3635384-21kworker/u8:2+flush-259:022:35:162
3427999992419,3cyclictest3687010-21/usr/sbin/munin23:20:142
3427999992419,3cyclictest3687010-21/usr/sbin/munin23:20:132
3427999992419,3cyclictest335557-21systemd-journal00:07:222
3427999992418,4cyclictest3619854-21latency_hist22:15:022
3427996992419,3cyclictest3443417-21latency_hist19:25:011
3427996992419,3cyclictest335557-21systemd-journal22:50:281
3427999992320,2cyclictest3728709-21kworker/u8:3+flush-259:000:10:232
3427999992320,2cyclictest3578242-21kworker/u8:1+events_unbound22:05:012
3427999992319,3cyclictest3448586-21kworker/u8:1+events_unbound19:45:132
3427999992318,4cyclictest3723353-21kworker/u8:2+events_unbound23:55:152
3427999992317,4cyclictest3697542-21latency_hist23:30:012
3427999992316,5cyclictest3546219-21ntpq21:00:202
3427999992316,5cyclictest335557-21systemd-journal22:30:012
3427992992319,3cyclictest3495755-21apt-get20:15:100
3427992992318,4cyclictest3753774-21ntpq00:20:220
3427992992317,4cyclictest3505801-21latency_hist20:25:000
3428003992219,2cyclictest507-21in:imuxsock20:20:003
3428003992216,4cyclictest335557-21systemd-journal20:05:003
3427999992215,5cyclictest3525609-21ntpq20:40:202
3427996992218,3cyclictest3661250-21/usr/sbin/munin22:55:151
3427996992218,3cyclictest3619892-21kworker/u8:0+flush-259:023:05:211
3427996992217,4cyclictest3448801-21/usr/sbin/munin19:30:131
3427992992219,2cyclictest3526753-21apt-get20:45:110
3427992992219,2cyclictest3448586-21kworker/u8:1+flush-259:020:10:010
3427992992217,3cyclictest3625044-21latency_hist22:20:010
3427992992216,4cyclictest3625464-21diskstats22:20:140
3428003992117,3cyclictest507-21in:imuxsock20:24:593
3428003992117,3cyclictest3739510-21apt-get00:10:113
3427999992118,2cyclictest3448586-21kworker/u8:1+events_unbound20:10:192
3427996992117,2cyclictest3516132-21sh20:34:591
3427996992116,4cyclictest3650996-21apt-get22:45:111
3427992992119,1cyclictest3578229-21latency_hist21:35:000
3427992992116,4cyclictest3655865-21kworker/u8:1+flush-259:022:55:250
3427992992116,3cyclictest3557786-21apt-get21:15:120
3427992992115,4cyclictest335557-21systemd-journal19:59:270
3428003992017,2cyclictest1-21systemd22:30:013
3428003992015,3cyclictest509-21rs:main3
3428003992014,4cyclictest3676472-21latency_hist23:10:013
3428003992011,1cyclictest0-21swapper/319:55:003
3427999992017,2cyclictest3692595-21kworker/u8:1+events_unbound23:45:272
3427999992017,2cyclictest3588777-21kworker/u8:4+flush-259:021:55:212
3427999992017,2cyclictest3433107-21sh19:15:002
3427999992016,3cyclictest3473529-21kworker/u8:0+flush-259:019:50:242
3427996992017,2cyclictest3448586-21kworker/u8:1+events_unbound20:55:181
3427992992018,1cyclictest0-21swapper/000:05:000
3427992992015,4cyclictest3578242-21kworker/u8:1+events_unbound21:50:000
3428003991912,5cyclictest3573043-21latency_hist21:30:003
3427999991915,3cyclictest3536868-21kworker/u8:3+events_unbound21:20:012
3427999991915,3cyclictest3495538-21kworker/u8:3+events_unbound20:20:202
3427999991914,3cyclictest3619892-21kworker/u8:0+flush-259:023:05:182
3427996991913,4cyclictest335557-21systemd-journal23:50:001
3427996991913,4cyclictest335557-21systemd-journal19:55:271
3427992991917,1cyclictest0-21swapper/022:55:000
3428003991814,2cyclictest1-21systemd21:35:013
3428003991813,3cyclictest335557-21systemd-journal00:39:293
3427999991816,1cyclictest3401017-21kworker/u8:3+events_unbound19:30:202
3427999991815,2cyclictest3536868-21kworker/u8:3+events_unbound21:20:182
3427999991814,3cyclictest3686763-21kworker/u8:0+events_unbound23:40:282
3427999991814,3cyclictest3681261-21kworker/u8:1+events_unbound23:15:152
3427999991814,3cyclictest3620083-21kworker/u8:3+events_unbound22:51:282
3427999991812,4cyclictest509-21rs:main2
3427999991811,5cyclictest3427926-21latency_hist19:10:012
3427996991815,2cyclictest3619892-21kworker/u8:0+flush-259:022:25:171
3427996991814,3cyclictest3686763-21kworker/u8:0+flush-259:023:35:231
3427996991814,3cyclictest3473529-21kworker/u8:0+events_unbound20:00:141
3427996991812,4cyclictest3760505-21apt-get00:30:101
3428003991712,3cyclictest3526495-21latency_hist20:45:003
3427999991714,2cyclictest3578242-21kworker/u8:1+events_unbound21:38:272
3427999991713,3cyclictest3448586-21kworker/u8:1+events_unbound20:05:012
3427999991713,3cyclictest3401017-21kworker/u8:3+events_unbound19:15:212
3427999991711,4cyclictest3537078-21/usr/sbin/munin20:55:122
3427996991711,4cyclictest3562942-21/usr/sbin/munin21:20:211
342799299178,8cyclictest0-21swapper/022:00:000
3427992991713,2cyclictest60350irq/132-enp0s29f1:rx-021:40:170
342799299170,15cyclictest1-21systemd19:39:270
3428003991613,1cyclictest0-21swapper/323:25:193
3427999991613,2cyclictest3588777-21kworker/u8:4+events_unbound21:45:162
3427999991613,2cyclictest3587648-21ntp_kernel_pll_21:40:212
3427999991612,3cyclictest3588777-21kworker/u8:4+flush-259:021:50:152
3427999991612,3cyclictest3448586-21kworker/u8:1+flush-259:020:25:012
3427999991611,3cyclictest3401017-21kworker/u8:3+flush-259:019:20:212
3427996991613,2cyclictest3693645-21kthreadcore23:25:171
3427996991610,4cyclictest3759996-21cat00:30:001
3427996991610,4cyclictest3671310-21latency_hist23:05:001
3427992991614,1cyclictest60350irq/132-enp0s29f1:rx-020:54:220
3427992991613,2cyclictest60350irq/132-enp0s29f1:rx-022:45:210
3427992991613,2cyclictest3578242-21kworker/u8:1+flush-259:021:40:000
3427992991613,1cyclictest0-21swapper/020:05:000
3427992991612,2cyclictest60350irq/132-enp0s29f1:rx-019:45:170
3427992991611,4cyclictest3697807-21apt-get23:30:110
3427992991610,4cyclictest335557-21systemd-journal22:35:280
342799299160,14cyclictest335557-21systemd-journal19:11:430
3428003991512,2cyclictest3617253-21kthreadcore22:10:173
3428003991511,3cyclictest509-21rs:main3
3428003991511,3cyclictest3438267-21kworker/u8:2+flush-259:019:25:243
3428003991511,3cyclictest335557-21systemd-journal21:18:273
3427999991513,1cyclictest3671541-21kworker/u8:4+events_unbound00:33:362
3427999991511,3cyclictest3754584-21kworker/u8:1+flush-259:000:25:262
3427999991510,4cyclictest3620083-21kworker/u8:3+flush-259:022:45:162
3427999991510,4cyclictest3619892-21kworker/u8:0+events_unbound22:31:282
342799999150,5cyclictest3754645-21basename00:25:012
342799699159,4cyclictest3536868-21kworker/u8:3+events_unbound21:15:001
342799699158,5cyclictest335557-21systemd-journal23:55:011
342799699158,5cyclictest335557-21systemd-journal22:00:011
342799699152,6cyclictest3645596-21munin-run22:40:001
3427996991512,2cyclictest3666166-21kworker/u8:2+events_unbound23:20:181
3427996991512,2cyclictest3666166-21kworker/u8:2+events_unbound23:20:171
3427996991512,2cyclictest3401017-21kworker/u8:3+events_unbound19:35:201
3427996991511,3cyclictest3609183-21kworker/u8:2+flush-259:022:10:221
3427996991511,3cyclictest3536868-21kworker/u8:3+events_unbound21:00:241
342799299158,5cyclictest3727591-21ntpq23:55:210
3427992991514,1cyclictest60350irq/132-enp0s29f1:rx-020:36:530
3427992991514,1cyclictest60350irq/132-enp0s29f1:rx-000:35:300
3427992991513,2cyclictest61150irq/133-enp0s29f1:tx-023:05:160
3427992991513,2cyclictest60350irq/132-enp0s29f1:rx-021:00:000
3427992991513,2cyclictest60350irq/132-enp0s29f1:rx-020:41:390
3427992991513,1cyclictest60350irq/132-enp0s29f1:rx-022:00:130
3427992991513,1cyclictest60350irq/132-enp0s29f1:rx-021:25:260
3427992991513,1cyclictest60350irq/132-enp0s29f1:rx-000:30:230
3427992991512,2cyclictest60350irq/132-enp0s29f1:rx-023:25:120
3427992991512,2cyclictest60350irq/132-enp0s29f1:rx-022:10:180
3427992991512,2cyclictest60350irq/132-enp0s29f1:rx-021:50:150
3427992991512,2cyclictest60350irq/132-enp0s29f1:rx-000:11:160
3427992991512,2cyclictest3552986-21irqcore21:10:170
3428003991412,1cyclictest0-21swapper/323:50:013
3428003991410,3cyclictest3401017-21kworker/u8:3+flush-259:019:10:323
3427999991411,2cyclictest3516161-21kworker/u8:0+events_unbound20:50:182
3427999991410,3cyclictest3686763-21kworker/u8:0+flush-259:023:50:182
3427999991410,3cyclictest3655865-21kworker/u8:1+events_unbound22:55:132
3427999991410,3cyclictest3619892-21kworker/u8:0+flush-259:022:20:212
3427999991410,3cyclictest3448586-21kworker/u8:1+events_unbound20:06:512
3427999991410,3cyclictest3438267-21kworker/u8:2+events_unbound19:25:182
3427996991411,2cyclictest3743932-21unixbench_singl00:10:241
3427996991411,2cyclictest3723353-21kworker/u8:2+flush-259:000:15:051
3427996991411,2cyclictest0-21swapper/123:20:001
3427996991410,3cyclictest3536868-21kworker/u8:3+events_unbound21:30:231
3427996991410,3cyclictest3448586-21kworker/u8:1+flush-259:019:50:211
3427992991413,1cyclictest60350irq/132-enp0s29f1:rx-023:04:460
3427992991413,1cyclictest60350irq/132-enp0s29f1:rx-019:25:510
3427992991412,1cyclictest60350irq/132-enp0s29f1:rx-023:35:230
3427992991412,1cyclictest60350irq/132-enp0s29f1:rx-023:10:170
3427992991412,1cyclictest60350irq/132-enp0s29f1:rx-020:25:240
3427992991412,1cyclictest60350irq/132-enp0s29f1:rx-019:40:280
3427992991412,1cyclictest0-21swapper/000:10:010
342800399138,3cyclictest335557-21systemd-journal00:30:363
3428003991311,1cyclictest3686763-21kworker/u8:0+flush-259:000:00:013
3428003991310,2cyclictest3625039-21cron22:20:003
3427999991310,2cyclictest3536868-21kworker/u8:3+events_unbound21:30:162
342799699138,4cyclictest3749238-21kworker/u8:0+events_unbound00:20:251
342799699138,3cyclictest335557-21systemd-journal19:15:011
3427996991311,1cyclictest61-21kcompactd023:55:181
3427996991310,2cyclictest3734176-21fschecks_time00:05:171
3427996991310,2cyclictest3583715-21cpuspeed_turbos21:40:141
3427996991310,2cyclictest0-21swapper/122:10:001
3427996991310,2cyclictest0-21swapper/100:40:011
3427992991311,1cyclictest60350irq/132-enp0s29f1:rx-023:42:530
3427992991310,2cyclictest60350irq/132-enp0s29f1:rx-000:15:200
342800399129,2cyclictest3681261-21kworker/u8:1+flush-259:023:18:563
342800399129,2cyclictest3676825-21cpuspeed_turbos23:10:133
342800399129,2cyclictest3609183-21kworker/u8:2+flush-259:022:05:253
342800399129,2cyclictest3516822-21irqcore20:35:163
342800399129,2cyclictest3448586-21kworker/u8:1+flush-259:020:25:163
342800399129,2cyclictest0-21swapper/323:25:013
342800399129,2cyclictest0-21swapper/323:25:003
3428003991210,1cyclictest0-21swapper/322:05:013
342799999129,2cyclictest3578242-21kworker/u8:1+events_unbound22:00:212
342799999129,2cyclictest3536868-21kworker/u8:3+events_unbound21:25:272
342799999126,4cyclictest3670404-21ntpq23:00:212
342799699129,2cyclictest3562245-21unixbench_multi21:15:271
342799699129,2cyclictest3459383-21fschecks_time19:40:141
342799699128,3cyclictest3448586-21kworker/u8:1+flush-259:020:20:161
342799699127,4cyclictest3433112-21latency_hist19:15:011
342799299129,2cyclictest60350irq/132-enp0s29f1:rx-023:45:130
342799299129,2cyclictest60350irq/132-enp0s29f1:rx-023:20:250
342799299129,2cyclictest60350irq/132-enp0s29f1:rx-023:20:250
342799299129,2cyclictest60350irq/132-enp0s29f1:rx-019:20:120
3427992991210,1cyclictest61150irq/133-enp0s29f1:tx-021:20:180
342800399118,2cyclictest3744817-21fschecks_count00:15:143
342800399118,2cyclictest3588877-21cpuspeed_turbos21:45:143
342800399118,2cyclictest3526819-21cpuspeed_turbos20:45:123
342799699119,1cyclictest0-21swapper/121:55:001
342799699118,1cyclictest0-21swapper/121:45:131
342799699117,3cyclictest3728940-21fschecks_count00:00:131
342800399108,1cyclictest0-21swapper/322:40:133
342800399108,1cyclictest0-21swapper/320:30:253
342800399107,2cyclictest3448586-21kworker/u8:1+flush-259:019:45:103
342800399107,2cyclictest3401017-21kworker/u8:3+flush-259:019:40:253
342800399107,2cyclictest0-21swapper/320:50:143
342800399106,2cyclictest3753746-21ntpq00:20:213
342800399106,2cyclictest3712146-21ntpq23:40:223
342799699107,2cyclictest3622429-21kthreadcore22:15:171
342799699106,3cyclictest3578242-21kworker/u8:1+flush-259:022:20:161
34280039996,2cyclictest3578736-21fschecks_time21:35:143
34279969995,3cyclictest311ktimers/123:10:111
34279969993,4cyclictest3604653-21apt-get22:00:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional