You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 13:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot3s.osadl.org (updated Sun Aug 23, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179472241167,24sleep30-21swapper/319:05:023
196832235162,60sleep00-21swapper/019:07:010
198952232162,57sleep10-21swapper/119:09:481
197872202168,22sleep20-21swapper/219:08:232
1138521850,1sleep12000099cyclictest23:40:231
2000599103100,2cyclictest0-21swapper/220:21:422
20005998177,3cyclictest0-21swapper/223:05:022
150892590,3sleep32001299cyclictest20:10:123
215002540,2sleep00-21swapper/000:05:150
183692520,4sleep123-21ksoftirqd/121:30:161
12032510,5sleep230-21rcuc/223:20:012
210852460,1sleep30-21swapper/300:05:013
262582440,1sleep30-21swapper/300:15:163
286932410,3sleep228695-21gltestperf20:40:182
2001299390,29cyclictest0-21swapper/323:45:243
19991993611,12cyclictest23484-21munin-run20:30:000
1999199345,13cyclictest20318-21munin-run22:50:000
1999199341,24cyclictest6149-21kworker/0:321:11:590
2001299330,25cyclictest0-21swapper/322:24:513
20005993326,2cyclictest0-21swapper/222:08:362
2001299310,24cyclictest0-21swapper/323:21:243
2001299310,20cyclictest0-21swapper/322:52:393
2001299301,10cyclictest0-21swapper/321:21:563
2001299300,24cyclictest0-21swapper/321:37:043
2001299300,23cyclictest0-21swapper/323:16:363
2001299300,22cyclictest0-21swapper/300:21:353
2001299291,16cyclictest0-21swapper/300:07:113
2001299291,10cyclictest0-21swapper/300:13:553
2001299291,10cyclictest0-21swapper/300:13:553
2001299290,13cyclictest0-21swapper/319:38:513
2000099295,10cyclictest28592-21tune2fs20:40:171
1999199292,12cyclictest16894-21awk20:15:000
1999199291,13cyclictest11537-21idleruntime-cro22:30:010
1999199290,14cyclictest22185-21basename19:15:000
194602290,3sleep230-21rcuc/222:45:232
2001299281,12cyclictest0-21swapper/320:45:313
2001299281,10cyclictest32497-21/usr/sbin/munin20:50:233
2001299281,10cyclictest0-21swapper/322:16:473
2001299281,10cyclictest0-21swapper/319:22:513
2001299281,10cyclictest0-21swapper/319:22:513
2001299280,13cyclictest0-21swapper/323:06:563
2001299271,11cyclictest0-21swapper/319:29:513
2001299270,13cyclictest0-21swapper/319:15:323
303612260,4sleep07-21ksoftirqd/019:30:170
2001299261,13cyclictest0-21swapper/323:25:243
2001299261,12cyclictest0-21swapper/323:04:203
2001299260,12cyclictest0-21swapper/322:49:033
2001299260,11cyclictest0-21swapper/322:42:433
2001299260,11cyclictest0-21swapper/319:43:273
2000099261,23cyclictest0-21swapper/121:25:121
2000099261,17cyclictest3742-21cstates23:25:121
1999199268,15cyclictest25690-21idleruntime-cro20:35:010
1999199261,12cyclictest24100-21munin-run21:45:020
2001299251,9cyclictest0-21swapper/323:40:123
2001299251,10cyclictest0-21swapper/321:56:193
2001299250,9cyclictest0-21swapper/323:31:443
2001299250,9cyclictest0-21swapper/322:34:473
2001299250,9cyclictest0-21swapper/321:32:523
2001299250,9cyclictest0-21swapper/319:30:153
2001299250,18cyclictest0-21swapper/321:45:563
2001299250,18cyclictest0-21swapper/321:45:553
19991992515,2cyclictest20648-21kworker/0:100:02:480
1999199251,19cyclictest31153-21kworker/0:023:13:230
1999199251,12cyclictest10295-21basename20:00:020
2001299241,9cyclictest0-21swapper/322:02:033
2001299241,9cyclictest0-21swapper/321:18:483
2001299241,11cyclictest0-21swapper/320:05:313
2001299241,10cyclictest3307-21sed00:35:253
2001299241,10cyclictest0-21swapper/320:35:513
2001299241,10cyclictest0-21swapper/319:45:593
2001299240,11cyclictest0-21swapper/322:28:033
2001299240,11cyclictest0-21swapper/321:52:273
2001299240,10cyclictest0-21swapper/321:25:243
2001299240,10cyclictest0-21swapper/321:10:203
2000099249,13cyclictest11430-21kworker/u8:023:33:061
2000099241,18cyclictest0-21swapper/122:49:501
2000099241,16cyclictest0-21swapper/123:24:111
2000099241,11cyclictest0-21swapper/122:30:171
2000099240,18cyclictest0-21swapper/122:44:351
1999199242,9cyclictest24978-21kworker/0:119:29:280
1999199242,8cyclictest14171-21kworker/0:123:53:020
1999199241,8cyclictest22049-21kworker/0:319:16:230
2001299236,3cyclictest0-21swapper/320:40:173
2001299231,3cyclictest0-21swapper/320:57:073
2001299231,13cyclictest0-21swapper/321:41:073
2001299231,12cyclictest0-21swapper/322:37:073
2001299231,11cyclictest0-21swapper/323:38:563
2001299231,11cyclictest0-21swapper/321:05:523
2001299230,9cyclictest0-21swapper/323:52:473
2001299230,9cyclictest0-21swapper/322:06:033
2001299230,10cyclictest0-21swapper/323:56:353
2001299230,10cyclictest0-21swapper/322:55:113
2001299230,10cyclictest0-21swapper/320:20:233
2000599231,4cyclictest0-21swapper/223:56:522
2000099233,16cyclictest7131-21perf22:20:021
2000099232,18cyclictest27076-21cat23:05:011
2000099232,14cyclictest26470-21sort00:15:201
20000992318,3cyclictest23484-21munin-run20:30:011
20000992317,3cyclictest12496-21munin-run20:05:021
2000099231,4cyclictest18060-21switchtime23:55:221
20000992311,9cyclictest0-21swapper/123:45:151
2000099231,16cyclictest0-21swapper/120:35:201
2000099231,14cyclictest31541-21/usr/sbin/munin23:15:111
20000992310,4cyclictest30256-21memory23:10:171
2000099230,7cyclictest0-21swapper/100:10:171
2000099230,7cyclictest0-21swapper/100:10:161
1999199237,7cyclictest12861-21kworker/0:121:22:150
1999199232,9cyclictest4242-21kworker/0:121:06:410
1999199232,9cyclictest17964-21kworker/0:123:09:210
1999199232,9cyclictest17312-21kworker/0:021:32:590
1999199232,8cyclictest25156-21kworker/0:000:14:040
1999199232,8cyclictest25156-21kworker/0:000:14:040
1999199232,7cyclictest31974-21kworker/0:122:04:360
1999199231,9cyclictest27182-21kworker/0:219:24:500
1999199231,9cyclictest27182-21kworker/0:219:24:490
259112220,5sleep01999199cyclictest00:15:140
2001299227,4cyclictest0-21swapper/320:25:153
2001299221,9cyclictest24693-21strings20:30:193
2001299221,13cyclictest0-21swapper/322:10:033
2001299221,10cyclictest0-21swapper/319:12:313
2001299220,8cyclictest0-21swapper/300:32:553
2001299220,10cyclictest0-21swapper/323:14:563
20005992211,3cyclictest5557-21awk23:30:012
2000099223,9cyclictest2729-21latency_hist22:10:021
2000099222,18cyclictest0-21swapper/119:17:001
2000099221,6cyclictest87350irq/126-enp1s0-23:50:131
2000099221,16cyclictest0-21swapper/120:08:391
2000099221,15cyclictest0-21swapper/100:38:321
20000992210,4cyclictest7376-21smart_sda19:50:251
1999199226,7cyclictest17964-21kworker/0:123:35:120
1999199225,15cyclictest21896-21munin-run21:40:000
1999199223,13cyclictest87350irq/126-enp1s0-21:25:150
1999199222,9cyclictest9195-21kworker/0:122:31:290
1999199222,9cyclictest14584-21kworker/0:020:51:170
1999199222,9cyclictest10096-21kworker/0:220:04:250
1999199222,8cyclictest30746-21kworker/0:019:36:420
1999199222,7cyclictest27392-21kworker/0:200:22:550
1999199222,11cyclictest5155-21/usr/sbin/munin22:15:120
1999199221,9cyclictest31974-21kworker/0:122:07:260
1999199221,9cyclictest22284-21kworker/0:022:54:090
1999199221,9cyclictest11848-21kworker/0:023:49:060
1999199221,8cyclictest21826-21kworker/0:121:47:100
1999199221,8cyclictest21826-21kworker/0:121:47:100
11122220,4sleep121-21rcuc/120:50:191
2001299211,11cyclictest808-21snmpd20:01:543
2001299210,11cyclictest0-21swapper/300:28:033
2000599212,7cyclictest27556-21munin-run00:20:012
2000599212,11cyclictest12812-21ntp_states22:30:212
20005992110,3cyclictest16543-21awk23:55:002
2000099215,7cyclictest0-21swapper/119:30:221
2000099214,14cyclictest3081-21cpuspeed_turbos22:10:131
2000099213,15cyclictest221ktimersoftd/121:45:221
2000099213,15cyclictest221ktimersoftd/121:45:221
2000099212,8cyclictest13942-21/usr/sbin/munin22:35:241
2000099212,2cyclictest221ktimersoftd/100:05:261
20000992112,7cyclictest31459-21kworker/u8:219:36:551
20000992112,2cyclictest87650irq/129-enp1s0-00:00:241
2000099211,18cyclictest0-21swapper/122:20:171
2000099211,18cyclictest0-21swapper/121:53:501
2000099211,18cyclictest0-21swapper/121:24:501
2000099211,17cyclictest8706-21diskmemload21:55:441
2000099211,17cyclictest1801-21cpuspeed_turbos19:40:121
2000099211,14cyclictest0-21swapper/120:21:141
2000099210,15cyclictest0-21swapper/122:55:261
1999199219,2cyclictest11848-21kworker/0:023:44:460
1999199218,3cyclictest4242-21kworker/0:121:02:070
1999199216,7cyclictest16456-21kworker/0:223:58:520
1999199212,9cyclictest29868-21kworker/0:220:45:090
1999199212,9cyclictest17964-21kworker/0:123:01:210
1999199212,9cyclictest14584-21kworker/0:020:16:410
1999199212,8cyclictest6955-21kworker/0:222:23:210
1999199212,8cyclictest31974-21kworker/0:122:12:330
1999199212,8cyclictest30746-21kworker/0:019:42:280
1999199212,8cyclictest23885-21kworker/0:221:58:420
1999199212,8cyclictest17964-21kworker/0:123:23:570
1999199212,8cyclictest12861-21kworker/0:121:17:570
1999199212,7cyclictest17964-21kworker/0:123:34:510
1999199212,7cyclictest15678-21kworker/0:222:44:270
19991992110,3cyclictest23088-21diskstats22:55:140
2000599201,3cyclictest87650irq/129-enp1s0-22:10:022
2000599201,11cyclictest0-21swapper/221:33:252
2000099206,12cyclictest21805-21kworker/u8:121:41:291
2000099205,9cyclictest21-21rcuc/121:05:181
2000099205,5cyclictest29905-21cut00:25:021
2000099202,3cyclictest0-21swapper/119:49:041
2000099202,16cyclictest0-21swapper/121:04:181
2000099202,15cyclictest29112-21latency_hist23:10:011
20000992016,3cyclictest0-21swapper/120:45:021
20000992016,3cyclictest0-21swapper/120:30:121
20000992014,4cyclictest0-21swapper/121:35:221
2000099201,18cyclictest0-21swapper/121:12:451
2000099201,17cyclictest0-21swapper/119:13:251
2000099201,14cyclictest0-21swapper/123:37:521
2000099201,11cyclictest27484-21/usr/sbin/munin19:25:141
2000099201,10cyclictest12350-21gzip21:15:231
1999199202,9cyclictest3211-21kworker/0:223:25:290
1999199202,7cyclictest27392-21kworker/0:200:26:470
1999199201,9cyclictest12770-21kworker/0:022:35:010
1999199201,8cyclictest17964-21kworker/0:123:19:050
1999199201,8cyclictest1342-21kworker/0:119:50:100
1999199200,9cyclictest14584-21kworker/0:020:20:370
1999199200,7cyclictest1342-21kworker/0:119:48:500
114292201,2sleep211566-21cat23:40:252
39642190,3sleep20-21swapper/220:55:522
2001299190,9cyclictest0-21swapper/319:55:193
2000599192,6cyclictest808-21snmpd20:55:022
2000599191,8cyclictest0-21swapper/221:18:572
2000599191,7cyclictest0-21swapper/222:28:392
2000599191,7cyclictest0-21swapper/200:10:322
2000599191,7cyclictest0-21swapper/200:10:322
2000599191,6cyclictest0-21swapper/223:01:402
2000599191,5cyclictest3514-40dnf19:42:562
2000599191,2cyclictest0-21swapper/221:23:502
2000099191,9cyclictest21320-21if_err_enp1s022:50:181
2000099191,17cyclictest0-21swapper/100:30:471
2000099191,16cyclictest0-21swapper/100:26:141
2000099191,14cyclictest0-21swapper/122:28:401
2000099191,10cyclictest15979-21sh20:10:211
2000099190,14cyclictest0-21swapper/122:00:121
2000099190,14cyclictest0-21swapper/120:19:071
1999199196,3cyclictest2007-21kworker/0:220:55:470
1999199194,3cyclictest87450irq/127-enp1s0-00:32:350
1999199192,7cyclictest26154-21kworker/0:021:51:160
1999199191,9cyclictest14584-21kworker/0:020:35:250
1999199191,10cyclictest1750-21kworker/0:200:39:210
238352181,3sleep20-21swapper/221:41:542
2001299181,12cyclictest0-21swapper/319:50:113
2000599187,3cyclictest87350irq/126-enp1s0-19:20:112
2000599187,3cyclictest87350irq/126-enp1s0-19:20:112
2000599183,3cyclictest808-21snmpd00:20:152
2000599182,6cyclictest87350irq/126-enp1s0-21:35:392
2000599181,8cyclictest14755-21irqstats22:35:192
2000599181,7cyclictest0-21swapper/223:20:232
2000599181,7cyclictest0-21swapper/221:57:042
2000599181,7cyclictest0-21swapper/200:26:052
2000599181,6cyclictest0-21swapper/221:26:142
2000599181,4cyclictest0-21swapper/223:35:252
20005991812,3cyclictest31650-21turbostat.cron19:35:012
20005991812,3cyclictest23460-21fschecks_count19:15:152
2000599181,10cyclictest755-21rngd22:00:152
20000991814,3cyclictest0-21swapper/120:55:351
2000099181,3cyclictest0-21swapper/119:20:531
2000099181,3cyclictest0-21swapper/119:20:531
20000991811,2cyclictest0-21swapper/119:56:301
1999199185,6cyclictest29868-21kworker/0:220:43:230
1999199181,9cyclictest808-21snmpd20:09:540
73462170,2sleep20-21swapper/222:20:102
274792171,2sleep20-21swapper/219:25:092
2001299171,14cyclictest0-21swapper/320:15:143
2000599175,3cyclictest14431-21sshd20:06:562
2000599174,4cyclictest23004-21cstates22:55:142
2000599171,9cyclictest0-21swapper/220:47:202
2000599171,7cyclictest21355-21irqstats22:50:202
2000599171,6cyclictest18435-21kworker/u8:000:02:402
2000599171,6cyclictest0-21swapper/223:34:202
2000599171,6cyclictest0-21swapper/222:18:582
2000599171,6cyclictest0-21swapper/200:38:382
2000599171,5cyclictest0-21swapper/221:07:222
2000599171,3cyclictest0-21swapper/223:46:032
2000599171,3cyclictest0-21swapper/221:50:272
20005991712,3cyclictest10508-21/usr/sbin/munin20:00:142
20005991712,2cyclictest6745-21gltestperf19:50:162
2000599171,15cyclictest0-21swapper/220:16:592
2000599171,10cyclictest0-21swapper/220:13:292
20005991710,4cyclictest25549-21/usr/sbin/munin21:45:222
20005991710,4cyclictest25549-21/usr/sbin/munin21:45:222
20005991710,4cyclictest22569-21ntpq20:25:212
2000599170,14cyclictest0-21swapper/219:55:232
16262171,2sleep20-21swapper/200:33:102
20012991610,3cyclictest4760-21expr21:00:153
2000599161,6cyclictest21885-21basename00:05:182
2000599161,5cyclictest0-21swapper/219:35:152
2000599161,4cyclictest0-21swapper/222:41:152
2000599161,2cyclictest0-21swapper/220:34:002
2000599161,14cyclictest0-21swapper/221:01:032
20005991611,3cyclictest25901-21/usr/sbin/munin20:35:242
20005991611,3cyclictest21603-21timerwakeupswit19:10:262
20005991611,3cyclictest10001-21ntp_states21:10:222
20005991610,3cyclictest29112-21latency_hist23:10:012
2000599152,11cyclictest4576-21cut19:45:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional