You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 04:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot6s.osadl.org (updated Wed Aug 26, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
276549910441042,2cyclictest15859-21kworker/3:1+events00:27:573
276489910411039,2cyclictest1456-21kworker/2:0+events21:44:332
276549910381036,2cyclictest23511-21turbostat21:54:553
276549910361033,1cyclictest18135-21kworker/3:1+events20:08:483
276549910311030,1cyclictest29241-21kworker/3:2+events23:01:063
276549910311029,2cyclictest19740-21kworker/3:1+events22:58:283
276549910271026,1cyclictest22129-21kworker/3:2+events20:26:043
276489910261023,3cyclictest14273-21turbostat00:19:552
276369910261023,2cyclictest21016-21kworker/0:1+events21:26:040
276549910251023,2cyclictest14294-21kworker/3:2+events22:42:363
276369910251024,1cyclictest5691-21kworker/0:1+events23:30:210
276549910241023,1cyclictest18135-21kworker/3:1+events20:04:353
276369910241023,1cyclictest5691-21kworker/0:1+events22:19:450
276369910221021,1cyclictest15480-21kworker/0:0+events21:22:060
276549910211019,2cyclictest4458-21turbostat20:54:553
276549910211019,2cyclictest1272-21kworker/3:2+events19:40:223
276549910201019,1cyclictest22158-21kworker/3:2+events00:32:113
276489910191017,2cyclictest7837-21kworker/2:2+events00:39:352
276549910181016,2cyclictest1272-21kworker/3:2+events19:23:043
276369910181017,1cyclictest5691-21kworker/0:1+events23:41:280
276549910171016,1cyclictest9608-21kworker/3:0+events00:16:073
276369910171015,2cyclictest28323-21turbostat20:34:540
276549910161015,1cyclictest7933-21kworker/3:0+events23:20:443
276549910161015,1cyclictest29241-21kworker/3:2+events23:06:283
276549910141012,2cyclictest19740-21kworker/3:1+events22:50:213
276549910111010,1cyclictest23869-21kworker/3:1+events22:13:193
276549910111010,1cyclictest13528-21kworker/3:1+events23:14:473
276549910101009,1cyclictest30861-21kworker/3:3+events00:11:063
276369910101007,3cyclictest1215-21turbostat22:34:560
276549910091008,1cyclictest18135-21kworker/3:1+events20:23:163
276549910091008,1cyclictest1272-21kworker/3:2+events19:33:043
276549910091007,2cyclictest15859-21kworker/3:1+events00:20:203
276549910081007,1cyclictest29970-21kworker/3:0+events19:16:263
276549910081007,1cyclictest23869-21kworker/3:1+events22:09:313
276549910071006,1cyclictest8675-21kworker/3:0+events22:49:023
276549910061005,1cyclictest30442-21kworker/3:1+events20:44:483
276549910061005,1cyclictest22129-21kworker/3:2+events20:34:123
276549910061005,1cyclictest13528-21kworker/3:1+events23:17:103
276549910061005,1cyclictest1272-21kworker/3:2+events19:36:053
276549910051004,1cyclictest7933-21kworker/3:0+events23:25:443
276549910051003,2cyclictest2132-21kworker/3:2+events21:49:443
276369910051001,3cyclictest5691-21kworker/0:1+events22:28:190
276369910041001,2cyclictest15480-21kworker/0:0+events21:13:590
276369910041001,1cyclictest16530-21kworker/0:0+events23:50:210
276549910031002,1cyclictest22129-21kworker/3:2+events20:36:053
276369910031002,1cyclictest32158-21kworker/0:0+events21:31:500
276549910021001,1cyclictest1272-21kworker/3:2+events19:54:023
276549910021001,1cyclictest1272-21kworker/3:2+events19:47:023
276549910011000,1cyclictest18135-21kworker/3:1+events20:11:083
27654991000999,1cyclictest2132-21kworker/3:2+events20:49:063
2765499998997,1cyclictest8675-21kworker/3:0+events22:37:573
2765499998997,1cyclictest22129-21kworker/3:2+events20:17:543
2765499998997,1cyclictest14020-21kworker/3:0+events19:55:233
2763699996996,0cyclictest6100-21kworker/0:2+events23:39:320
2763699996995,1cyclictest18397-21kworker/0:2+events22:21:470
2763699994993,1cyclictest7132-21kworker/0:1+events20:57:090
2763699994993,1cyclictest6100-21kworker/0:2+events23:57:110
2763699994993,1cyclictest15480-21kworker/0:0+events21:16:530
2763699992991,1cyclictest6100-21kworker/0:2+events00:00:200
2763699991990,1cyclictest5691-21kworker/0:1+events21:59:460
2763699990989,1cyclictest27653-21kworker/0:2+events21:08:210
2763699989987,2cyclictest14169-21turbostat19:59:550
2764299988987,1cyclictest1914-21kworker/1:1+events19:29:111
2763699988985,2cyclictest5691-21kworker/0:1+events23:45:210
2763699987983,2cyclictest7003-21kworker/0:1+events00:09:080
2763699986985,1cyclictest3964-21kworker/0:0+events22:01:050
2763699984983,1cyclictest7132-21kworker/0:1+events21:01:490
2765499979976,3cyclictest25363-21turbostat23:59:553
2764899978977,1cyclictest27475-21kworker/2:1+events19:11:062
2763699978975,2cyclictest27509-21turbostat19:14:540
2764899968967,1cyclictest7837-21kworker/2:2+events21:39:532
2764299959957,2cyclictest1418-21turbostat21:34:551
2763699951949,2cyclictest1656-21turbostat19:29:550
2764299933931,2cyclictest30335-21turbostat00:34:561
2763699887884,3cyclictest10146-21turbostat19:49:550
2764899863861,2cyclictest1539-21turbostat22:04:552
2763699863861,2cyclictest12682-21turbostat21:44:540
2765499855853,2cyclictest3150-21turbostat00:39:553
2763699673670,3cyclictest26107-21turbostat23:29:550
2764899671667,4cyclictest30335-21turbostat00:34:552
2765499656654,2cyclictest1539-21turbostat22:04:553
2765499635633,2cyclictest12682-21turbostat21:44:553
2764299623620,3cyclictest32015-21turbostat19:24:551
2763699593591,2cyclictest3150-21turbostat00:39:550
2765499580578,2cyclictest1215-21turbostat22:34:563
2763699576574,2cyclictest23511-21turbostat21:54:550
2763699560557,3cyclictest4458-21turbostat20:54:550
2765499548546,2cyclictest3438-21turbostat00:09:553
2765499545544,1cyclictest27509-21turbostat19:14:553
2763699524522,1cyclictest12364-21turbostat22:14:550
2764899522521,1cyclictest1418-21turbostat21:34:552
2765499521520,1cyclictest1656-21turbostat19:29:553
27654992120,1cyclictest2132-21kworker/3:2+events_freezable21:20:503
27654992019,1cyclictest67750irq/123-lan23:40:123
27654991919,0cyclictest67750irq/123-lan23:37:473
27654991919,0cyclictest67750irq/123-lan22:26:193
27654991919,0cyclictest67750irq/123-lan21:55:123
27654991919,0cyclictest67750irq/123-lan21:27:083
27654991918,1cyclictest67750irq/123-lan23:30:183
27654991918,1cyclictest67750irq/123-lan22:22:263
27654991818,0cyclictest784-21snmpd22:17:543
27654991818,0cyclictest784-21snmpd00:02:223
27654991818,0cyclictest67750irq/123-lan21:30:013
27654991817,1cyclictest67750irq/123-lan23:50:133
27654991817,1cyclictest67750irq/123-lan23:49:243
27654991717,0cyclictest67750irq/123-lan21:39:323
27654991717,0cyclictest67750irq/123-lan21:02:493
27654991716,1cyclictest67750irq/123-lan21:15:173
27654991716,1cyclictest2132-21kworker/3:2+events21:10:093
27654991615,1cyclictest67750irq/123-lan21:05:253
27654991413,1cyclictest67750irq/123-lan20:55:173
276489998,1cyclictest784-21snmpd21:54:302
276429998,1cyclictest784-21snmpd21:01:201
276429998,1cyclictest782-21lldpd22:01:521
276429998,1cyclictest526-21avahi-daemon23:48:511
276429998,1cyclictest526-21avahi-daemon23:35:001
276429998,1cyclictest526-21avahi-daemon19:58:291
276429996,3cyclictest762-21ntpd22:45:551
276369997,2cyclictest16431-21sed23:15:130
276489988,0cyclictest784-21snmpd22:20:132
276489988,0cyclictest784-21snmpd20:21:372
276489988,0cyclictest782-21lldpd22:28:282
276489988,0cyclictest782-21lldpd21:10:092
276489988,0cyclictest782-21lldpd00:01:282
276489987,1cyclictest784-21snmpd23:28:212
276489987,1cyclictest782-21lldpd20:43:172
276489987,1cyclictest526-21avahi-daemon23:51:062
276489986,2cyclictest784-21snmpd23:43:312
276429988,0cyclictest784-21snmpd21:11:261
276429988,0cyclictest784-21snmpd20:29:101
276429988,0cyclictest782-21lldpd22:59:031
276429988,0cyclictest782-21lldpd21:48:341
276429988,0cyclictest782-21lldpd20:54:471
276429988,0cyclictest526-21avahi-daemon19:19:211
276429987,1cyclictest782-21lldpd20:35:431
276429986,2cyclictest784-21snmpd22:50:381
276429986,2cyclictest784-21snmpd22:25:191
276429986,2cyclictest784-21snmpd20:03:301
276429986,2cyclictest762-21ntpd23:57:361
276429986,2cyclictest762-21ntpd23:52:361
276429986,2cyclictest762-21ntpd23:30:321
276429986,2cyclictest762-21ntpd23:26:101
276429986,2cyclictest762-21ntpd23:20:381
276429986,2cyclictest762-21ntpd23:11:241
276429986,2cyclictest762-21ntpd23:00:311
276429986,2cyclictest762-21ntpd22:40:391
276429986,2cyclictest762-21ntpd22:35:351
276429986,2cyclictest762-21ntpd22:30:211
276429986,2cyclictest762-21ntpd22:21:371
276429986,2cyclictest762-21ntpd22:15:511
276429986,2cyclictest762-21ntpd22:10:091
276429986,2cyclictest762-21ntpd22:07:371
276429986,2cyclictest762-21ntpd21:55:061
276429986,2cyclictest762-21ntpd21:18:361
276429986,2cyclictest762-21ntpd20:10:131
276429986,2cyclictest762-21ntpd00:17:551
276429986,2cyclictest762-21ntpd00:10:281
276429986,2cyclictest762-21ntpd00:06:141
276429986,2cyclictest762-21ntpd00:00:541
276429984,3cyclictest25693-21diskstats00:25:131
276369988,0cyclictest784-21snmpd23:22:210
276369988,0cyclictest784-21snmpd20:16:400
276369988,0cyclictest782-21lldpd23:10:390
276369988,0cyclictest782-21lldpd20:00:260
276369988,0cyclictest782-21lldpd19:38:480
276369988,0cyclictest782-21lldpd00:29:320
276369987,1cyclictest784-21snmpd22:42:140
276369987,1cyclictest784-21snmpd21:46:270
276369987,1cyclictest784-21snmpd20:37:190
276369986,2cyclictest784-21snmpd19:41:450
276369986,2cyclictest762-21ntpd23:06:470
276369986,2cyclictest762-21ntpd22:50:150
276369986,2cyclictest762-21ntpd22:45:430
276369986,2cyclictest762-21ntpd22:06:590
276369986,2cyclictest762-21ntpd21:37:200
276369986,2cyclictest762-21ntpd19:50:140
276369986,2cyclictest762-21ntpd00:31:090
276369986,2cyclictest762-21ntpd00:18:190
276369986,2cyclictest762-21ntpd00:11:420
276489977,0cyclictest784-21snmpd19:54:422
276489977,0cyclictest784-21snmpd19:31:262
276489977,0cyclictest784-21snmpd00:26:172
276489977,0cyclictest782-21lldpd23:39:432
276489977,0cyclictest782-21lldpd23:17:272
276489977,0cyclictest782-21lldpd20:16:272
276489976,1cyclictest784-21snmpd23:32:032
276489976,1cyclictest784-21snmpd20:57:292
276489976,1cyclictest784-21snmpd20:25:212
276489976,1cyclictest784-21snmpd20:03:572
276489976,1cyclictest526-21avahi-daemon23:07:482
276489976,1cyclictest25999-21fschecks_time23:55:132
276489976,1cyclictest245-20systemd-journal21:24:422
276489976,1cyclictest20712-21gltestperf00:20:142
276489975,2cyclictest784-21snmpd22:09:332
276489975,2cyclictest784-21snmpd20:30:372
276489975,2cyclictest784-21snmpd19:24:052
276489974,3cyclictest547-21NetworkManager19:25:172
276489973,3cyclictest14229-21ssh23:13:002
276429977,0cyclictest784-21snmpd20:59:381
276429977,0cyclictest784-21snmpd20:21:101
276429977,0cyclictest784-21snmpd20:07:311
276429977,0cyclictest782-21lldpd21:50:021
276429977,0cyclictest782-21lldpd21:40:161
276429977,0cyclictest782-21lldpd20:31:091
276429977,0cyclictest782-21lldpd19:31:411
276429976,1cyclictest784-21snmpd19:53:531
276429976,1cyclictest784-21snmpd00:39:031
276429976,1cyclictest762-21ntpd23:16:101
276429976,1cyclictest762-21ntpd00:20:011
276429975,2cyclictest784-21snmpd21:07:051
276429975,2cyclictest762-21ntpd23:40:341
276429975,2cyclictest762-21ntpd21:36:101
276429975,2cyclictest762-21ntpd21:26:161
276429975,2cyclictest762-21ntpd21:20:031
276429973,3cyclictest23213-21ntpq20:15:171
276369977,0cyclictest784-21snmpd22:36:070
276369977,0cyclictest784-21snmpd20:40:200
276369977,0cyclictest782-21lldpd19:34:270
276369976,1cyclictest784-21snmpd19:15:350
276369976,1cyclictest762-21ntpd23:01:170
276369976,1cyclictest762-21ntpd22:55:230
276369976,1cyclictest24514-21/usr/sbin/munin20:20:180
276369975,2cyclictest784-21snmpd20:26:340
276489966,0cyclictest782-21lldpd22:36:232
276489966,0cyclictest782-21lldpd21:29:582
276489966,0cyclictest782-21lldpd00:14:382
276489965,1cyclictest784-21snmpd23:46:552
276489965,1cyclictest30459-21fschecks_count19:15:142
276489965,1cyclictest22681-21fschecks_time22:50:142
276489964,2cyclictest784-21snmpd23:02:232
276489964,2cyclictest784-21snmpd22:31:402
276489964,2cyclictest784-21snmpd19:45:212
276489964,2cyclictest547-21NetworkManager23:24:242
276489963,2cyclictest245-20systemd-journal22:11:392
276489963,2cyclictest17378-21ssh22:45:122
276489962,3cyclictest8856-21ssh00:09:442
276489962,3cyclictest31307-21fw_forwarded_lo20:35:142
276489962,3cyclictest21424-21ssh21:47:062
276489962,3cyclictest20914-21ssh22:16:512
276429966,0cyclictest782-21lldpd19:39:021
276429964,2cyclictest784-21snmpd19:13:581
276429963,3cyclictest804-21df20:40:121
276429963,2cyclictest5816-21df_inode23:05:111
276429962,3cyclictest3428-21ld20:45:171
276369964,2cyclictest784-21snmpd19:24:020
276369962,3cyclictest23901-21ssh00:23:190
276489955,0cyclictest8711-21head19:40:132
276489953,2cyclictest547-21NetworkManager19:58:092
276489952,3cyclictest547-21NetworkManager21:00:472
276489952,2cyclictest30632-21ssh21:55:272
276489952,2cyclictest30016-21ssh22:56:372
276489952,2cyclictest18549-21ssh21:15:122
276489952,2cyclictest14865-21ssh22:42:162
276489951,3cyclictest2821-21df20:45:132
276489951,3cyclictest18916-21awk20:05:142
276489950,4cyclictest6695-21ntpq19:35:172
276429954,1cyclictest0-21swapper/119:45:171
276429950,4cyclictest9254-21uname19:40:181
276369955,0cyclictest18860-21gltestperf20:05:150
276369954,1cyclictest762-21ntpd20:45:250
276369952,2cyclictest21263-21ld20:10:170
276489943,1cyclictest20785-21gltestperf20:10:152
276489941,2cyclictest5547-21idleruntime20:50:192
276489940,3cyclictest11375-21/usr/sbin/munin21:05:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional