You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-23 - 18:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Thu Jul 23, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
67732240163,25sleep20-21swapper/207:06:192
68972238163,24sleep30-21swapper/307:07:573
67352238163,24sleep10-21swapper/107:05:481
67762236161,26sleep00-21swapper/007:06:210
921321830,4sleep1715099cyclictest12:05:201
2557221630,4sleep1715099cyclictest10:15:201
336721380,3sleep30-21swapper/311:50:333
1095221230,2sleep30-21swapper/307:15:183
7157997976,2cyclictest0-21swapper/209:13:062
7157996865,2cyclictest0-21swapper/207:51:532
7157996865,2cyclictest0-21swapper/207:51:532
64332550,3sleep00-21swapper/012:00:130
259792550,4sleep011-21rcuc/011:30:140
7157994946,2cyclictest0-21swapper/211:09:462
7157993836,1cyclictest0-21swapper/207:44:092
714499368,13cyclictest14732-21munin-run12:20:010
7157993532,2cyclictest0-21swapper/210:33:342
714499349,12cyclictest11247-21munin-run09:45:020
716199313,12cyclictest0-21swapper/310:24:173
714499311,14cyclictest27602-21awk07:55:010
714499311,14cyclictest27602-21awk07:55:010
65782300,8sleep339-21rcuc/308:20:123
75472290,1sleep10-21swapper/112:00:251
715799291,11cyclictest0-21swapper/212:15:002
715099290,6cyclictest17443-21/usr/sbin/munin08:45:241
715799281,7cyclictest0-21swapper/207:21:522
715799281,13cyclictest0-21swapper/211:53:172
715799281,10cyclictest0-21swapper/211:41:092
715099281,18cyclictest13019-21unixbench_singl09:45:271
715799271,8cyclictest0-21swapper/212:00:122
715799270,11cyclictest0-21swapper/211:39:412
715799261,10cyclictest0-21swapper/209:57:042
714499265,10cyclictest0-21swapper/011:25:000
715799251,11cyclictest2156-21ntp_states10:35:212
714499251,13cyclictest16622-21basename07:30:000
7144992510,6cyclictest0-21swapper/012:30:110
716199247,7cyclictest0-21swapper/310:28:243
716199247,5cyclictest27938-21cut11:35:163
715799241,9cyclictest0-21swapper/211:29:172
715799241,9cyclictest0-21swapper/211:29:172
715799241,9cyclictest0-21swapper/208:44:522
715799241,10cyclictest0-21swapper/212:38:002
715799240,9cyclictest0-21swapper/212:08:282
715099242,19cyclictest31454-21irqstats09:15:191
715099241,19cyclictest11865-21munin-run11:00:021
715099241,12cyclictest0-21swapper/112:21:441
715099241,11cyclictest0-21swapper/109:21:401
715099241,11cyclictest0-21swapper/109:12:131
7150992411,11cyclictest28440-21kworker/u8:209:56:001
714499247,15cyclictest17848-21perf10:00:010
714499241,11cyclictest29787-21munin-run08:00:010
7144992410,3cyclictest24329-21kworker/0:110:15:150
716199237,2cyclictest0-21swapper/309:00:153
716199231,6cyclictest0-21swapper/312:37:343
716199230,11cyclictest0-21swapper/311:29:083
716199230,11cyclictest0-21swapper/311:29:083
715799231,6cyclictest0-21swapper/210:52:242
715799230,12cyclictest0-21swapper/211:22:212
715099232,15cyclictest0-21swapper/111:38:221
715099232,12cyclictest2008-21memory10:35:181
7150992317,4cyclictest24421-21switchtime11:25:241
7150992317,4cyclictest24421-21switchtime11:25:241
715099231,16cyclictest0-21swapper/111:48:081
715099231,11cyclictest18620-21sed12:25:251
714499237,4cyclictest9127-21kworker/0:107:15:040
714499234,16cyclictest18806-21munin-run07:35:000
714499231,10cyclictest32402-21kworker/0:109:32:280
91412220,8sleep123-21ksoftirqd/108:25:161
716199224,7cyclictest0-21swapper/309:14:473
7161992211,3cyclictest13447-21awk09:50:023
7157992216,3cyclictest11480-21find07:15:252
715799221,13cyclictest0-21swapper/208:09:272
715799220,11cyclictest0-21swapper/211:59:452
715099229,4cyclictest0-21swapper/111:56:221
715099222,18cyclictest0-21swapper/107:43:011
715099221,8cyclictest579-21systemd-journal08:35:021
7150992214,3cyclictest0-21swapper/112:19:301
715099221,3cyclictest5295-21fschecks_count09:30:151
715099221,3cyclictest0-21swapper/110:07:211
7150992213,2cyclictest24362-21sshd10:14:161
715099221,16cyclictest0-21swapper/111:53:001
715099221,13cyclictest0-21swapper/111:00:161
715099221,11cyclictest0-21swapper/110:21:321
715099221,10cyclictest0-21swapper/109:36:361
715099221,10cyclictest0-21swapper/109:25:311
714499229,3cyclictest3650-21kworker/0:011:56:070
714499226,14cyclictest0-21swapper/010:15:010
714499224,3cyclictest0-21swapper/007:20:190
716199217,6cyclictest0-21swapper/311:31:383
716199213,2cyclictest0-21swapper/309:23:593
716199212,11cyclictest17443-21/usr/sbin/munin08:45:183
7161992116,3cyclictest30363-21tail11:40:173
715099218,3cyclictest0-21swapper/110:46:451
715099212,3cyclictest0-21swapper/108:59:311
715099211,3cyclictest827-21runrttasks11:15:101
715099211,18cyclictest0-21swapper/108:10:031
715099211,17cyclictest32428-21ps10:30:231
715099211,15cyclictest0-21swapper/111:10:571
715099211,15cyclictest0-21swapper/111:07:521
715099211,12cyclictest0-21swapper/112:14:061
715099210,4cyclictest0-21swapper/107:55:181
714499216,3cyclictest6644-21kworker/0:010:05:110
714499216,12cyclictest31042-21basename10:30:010
714499215,2cyclictest87550irq/128-enp1s0-09:07:100
714499215,2cyclictest87550irq/128-enp1s0-09:07:100
714499211,11cyclictest15627-21hddtemp_smartct12:20:160
26672210,5sleep011-21rcuc/010:35:250
716199209,8cyclictest10402-21sshd08:27:153
716199209,3cyclictest23422-21/usr/sbin/munin07:45:253
716199202,8cyclictest18245-1kworker/u9:210:03:313
716199202,3cyclictest0-21swapper/308:00:043
716199202,10cyclictest3289-21/usr/sbin/munin10:40:263
716199201,12cyclictest0-21swapper/312:25:233
716199201,12cyclictest0-21swapper/309:40:203
716199201,12cyclictest0-21swapper/307:35:173
716199201,11cyclictest24659-21/usr/sbin/munin10:15:133
716199201,11cyclictest0-21swapper/311:56:313
716199201,10cyclictest5865-21strings09:30:203
716199200,9cyclictest0-21swapper/311:09:253
716199200,11cyclictest0-21swapper/312:10:163
7157992017,2cyclictest0-21swapper/208:03:542
715799201,13cyclictest0-21swapper/210:40:252
715099202,10cyclictest31780-21chrt08:02:231
7150992017,2cyclictest0-21swapper/109:00:271
7150992017,2cyclictest0-21swapper/108:53:061
7150992016,3cyclictest0-21swapper/108:43:411
7150992015,3cyclictest0-21swapper/107:47:121
7150992013,3cyclictest0-21swapper/111:22:531
7150992012,3cyclictest0-21swapper/111:42:471
715099201,13cyclictest0-21swapper/110:50:401
715099201,11cyclictest580-21irqstats08:05:181
715099201,11cyclictest21878-21df12:35:121
715099201,11cyclictest19844-21irqstats07:35:171
715099201,11cyclictest10711-21ls09:40:241
715099201,11cyclictest0-21swapper/110:40:391
715099200,18cyclictest0-21swapper/107:20:141
714499206,4cyclictest9976-21kworker/0:012:27:230
714499204,3cyclictest87450irq/127-enp1s0-11:00:420
714499201,3cyclictest713-21expr09:20:110
714499201,11cyclictest29719-21/usr/sbin/munin11:40:250
714499201,11cyclictest19032-21/usr/sbin/munin07:35:260
131722200,4sleep011-21rcuc/008:35:140
716199195,8cyclictest14191-40dnf07:22:253
716199194,4cyclictest0-21swapper/312:15:323
716199193,9cyclictest0-21swapper/311:10:463
716199191,8cyclictest23330-21unixbench_multi08:55:283
7161991914,3cyclictest14909-21find09:50:243
716199191,12cyclictest0-21swapper/309:20:003
716199191,10cyclictest0-21swapper/312:30:173
716199191,10cyclictest0-21swapper/312:09:443
716199191,10cyclictest0-21swapper/310:52:583
716199191,10cyclictest0-21swapper/310:38:293
716199191,10cyclictest0-21swapper/309:59:193
715799191,8cyclictest13716-21grep08:35:172
715799191,8cyclictest0-21swapper/209:19:412
715799191,3cyclictest87350irq/126-enp1s0-12:25:112
715099198,3cyclictest17173-21df_inode07:30:121
715099196,3cyclictest15018-21latency_hist08:40:011
715099192,16cyclictest0-21swapper/108:17:561
7150991912,3cyclictest0-21swapper/109:53:041
715099191,17cyclictest0-21swapper/111:30:351
715099191,17cyclictest0-21swapper/107:12:051
715099191,16cyclictest0-21swapper/108:20:411
714499199,2cyclictest16491-21/usr/sbin/munin11:10:230
714499196,3cyclictest2932-21kworker/0:210:40:430
714499192,6cyclictest1-21systemd07:40:170
714499192,3cyclictest16581-21kworker/0:312:36:550
714499191,5cyclictest0-21swapper/009:35:210
714499191,10cyclictest4256-21/usr/sbin/munin08:15:160
714499191,10cyclictest11472-21/usr/sbin/munin09:45:230
307222191,2sleep10-21swapper/110:26:161
281672191,15sleep328371-21tune2fs07:55:173
116712190,2sleep20-21swapper/210:57:442
716199187,5cyclictest0-21swapper/311:04:573
716199185,3cyclictest0-21swapper/309:39:213
716199181,7cyclictest434-21sshd08:05:173
716199181,7cyclictest23273-21cat07:45:013
716199181,7cyclictest11203-21diskstats08:30:133
716199181,6cyclictest0-21swapper/312:21:523
716199181,6cyclictest0-21swapper/311:15:403
716199181,6cyclictest0-21swapper/310:58:483
716199181,4cyclictest28210-21diskmemload09:25:273
716199181,11cyclictest0-21swapper/312:03:073
716199181,10cyclictest16832-21/usr/sbin/munin07:30:223
716199181,10cyclictest0-21swapper/311:45:223
716199181,10cyclictest0-21swapper/310:46:453
716199181,10cyclictest0-21swapper/310:32:563
715799187,3cyclictest827-21runrttasks08:45:052
715799182,5cyclictest87550irq/128-enp1s0-12:33:062
715799181,9cyclictest24440-21awk10:15:002
715799181,6cyclictest0-21swapper/211:15:142
715799181,6cyclictest0-21swapper/210:46:572
715799181,6cyclictest0-21swapper/209:53:412
715799181,6cyclictest0-21swapper/209:41:212
715799181,5cyclictest0-21swapper/210:26:232
715799181,3cyclictest0-21swapper/210:23:592
7157991812,4cyclictest31702-21kworker/u8:012:15:322
7157991812,3cyclictest15221-21gltestperf07:25:162
715099186,5cyclictest0-21swapper/110:00:021
715099182,14cyclictest0-21swapper/107:25:161
715099181,4cyclictest0-21swapper/112:33:201
7150991813,2cyclictest25743-21cpuspeed_turbos07:50:121
7150991813,2cyclictest25743-21cpuspeed_turbos07:50:121
715099181,16cyclictest0-21swapper/109:06:461
715099181,16cyclictest0-21swapper/109:06:461
715099181,16cyclictest0-21swapper/107:17:231
714499187,3cyclictest5488-21/usr/sbin/munin10:45:190
714499186,3cyclictest20655-21kworker/0:211:35:220
714499186,11cyclictest0-21swapper/007:15:020
714499185,4cyclictest87550irq/128-enp1s0-12:10:280
714499185,3cyclictest87450irq/127-enp1s0-09:15:370
714499183,2cyclictest87350irq/126-enp1s0-12:05:150
714499182,13cyclictest28574-21tr09:10:120
7144991812,3cyclictest14946-21swap09:50:250
714499181,14cyclictest32535-21head11:45:160
7144991811,4cyclictest18832-21cpuspeed_turbos11:15:110
215092181,2sleep20-21swapper/210:05:212
716199178,2cyclictest0-21swapper/310:05:123
716199174,2cyclictest0-21swapper/307:10:133
716199173,8cyclictest20531-21kworker/u8:011:20:103
716199171,8cyclictest0-21swapper/307:25:193
716199171,6cyclictest0-21swapper/310:10:343
7161991713,3cyclictest0-21swapper/308:51:193
716199171,13cyclictest579-21systemd-journal08:10:173
7161991710,4cyclictest15907-21gltestperf08:40:153
715799178,2cyclictest87550irq/128-enp1s0-09:00:152
715799174,3cyclictest12050irq/123-ahci[0009:35:022
715799173,8cyclictest0-21swapper/211:13:242
715799173,4cyclictest685-21ps11:45:232
715799172,6cyclictest2677-21/usr/sbin/munin09:25:152
715799172,4cyclictest13450-21munin-run09:50:012
715799171,9cyclictest25243-21sshd07:48:162
715799171,6cyclictest4462-21expr08:15:122
715799171,6cyclictest29948-21cut08:00:022
715799171,6cyclictest0-21swapper/212:22:112
715799171,5cyclictest0-21swapper/207:34:152
7157991713,3cyclictest0-21swapper/208:21:462
7157991712,3cyclictest27692-21sed09:05:252
7157991712,3cyclictest27692-21sed09:05:252
715799170,15cyclictest0-21swapper/211:30:302
714499174,2cyclictest27816-21kworker/0:308:00:320
714499173,2cyclictest87550irq/128-enp1s0-08:59:500
714499171,9cyclictest14953-21kworker/0:108:52:570
714499171,9cyclictest11527-21kworker/0:310:57:260
714499171,6cyclictest0-21swapper/011:50:320
714499171,4cyclictest25034-21kworker/0:007:47:350
7144991712,3cyclictest14299-21/usr/sbin/munin11:05:170
7144991711,3cyclictest6799-21dnf08:20:140
714499171,10cyclictest0-21swapper/009:25:190
7144991710,4cyclictest0-21swapper/008:05:190
285742170,2sleep00-21swapper/010:21:540
198922171,2sleep00-21swapper/010:03:160
7161991612,3cyclictest0-21swapper/309:09:223
7161991612,3cyclictest0-21swapper/309:09:223
7161991612,3cyclictest0-21swapper/308:15:153
7161991611,3cyclictest14458-21timerwakeupswit08:35:263
715799165,3cyclictest87350irq/126-enp1s0-08:50:222
715799163,3cyclictest21102-21ls07:40:012
715799161,6cyclictest19323-21sensors10:00:242
715799161,6cyclictest12441-21dnf11:00:152
715799161,6cyclictest0-21swapper/209:20:112
715799161,5cyclictest23364-21grep08:55:282
715799161,2cyclictest0-21swapper/210:15:142
715799161,14cyclictest0-21swapper/207:13:442
7157991611,3cyclictest827-21runrttasks09:36:302
7157991611,3cyclictest12085-21gzip08:30:232
715799161,12cyclictest9130-21cut08:25:152
714499169,4cyclictest19004-21unixbench_singl08:45:240
714499161,5cyclictest0-21swapper/010:33:020
7144991612,3cyclictest0-21swapper/008:10:060
7144991611,3cyclictest8806-21cut08:25:130
7144991611,3cyclictest23127-21/usr/sbin/munin11:25:180
7144991611,3cyclictest23127-21/usr/sbin/munin11:25:180
7144991611,3cyclictest15941-21hostname08:40:160
7144991611,3cyclictest11524-21head08:30:160
714499161,12cyclictest24655-21gltestperf09:00:150
7144991610,3cyclictest7691-21/usr/sbin/munin10:50:240
716199150,5cyclictest0-21swapper/307:50:183
716199150,5cyclictest0-21swapper/307:50:183
715799150,2cyclictest87550irq/128-enp1s0-08:10:162
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional