You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 19:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3s.osadl.org (updated Sat May 09, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
35182238163,24sleep00-21swapper/007:09:350
32122211179,21sleep30-21swapper/307:05:363
33602202165,24sleep10-21swapper/107:07:311
15902201164,24sleep20-21swapper/207:05:112
1992021740,4sleep0363699cyclictest11:25:150
36489910299,2cyclictest0-21swapper/207:42:092
3648997973,3cyclictest3155-21taskset09:32:552
289132550,1sleep10-21swapper/111:45:141
94512520,2sleep011-21rcuc/007:20:140
58342470,1sleep10-21swapper/109:40:131
318382470,1sleep230-21rcuc/209:25:132
74792410,1sleep10-21swapper/110:55:221
364899410,29cyclictest0-21swapper/210:46:382
364899351,25cyclictest18181-21irqstats11:20:172
363699357,13cyclictest22507-21munin-run09:05:000
364899341,12cyclictest0-21swapper/208:47:582
321532330,1sleep30-21swapper/310:40:113
364899321,13cyclictest23729-21latency_hist11:35:012
364899321,13cyclictest0-21swapper/207:50:212
364899311,11cyclictest19536-21awk08:55:222
363699318,11cyclictest1762-21perf10:45:010
364899301,19cyclictest0-21swapper/208:11:332
364899301,13cyclictest2365-21turbostat.cron12:00:012
364899300,13cyclictest0-21swapper/210:37:262
364399309,12cyclictest0-21swapper/109:10:221
363699302,13cyclictest7132-21idleruntime-cro08:30:010
364899291,13cyclictest0-21swapper/208:44:582
364899291,11cyclictest0-21swapper/212:18:052
364899290,12cyclictest0-21swapper/210:10:252
364899280,8cyclictest0-21swapper/207:18:222
364899280,11cyclictest0-21swapper/207:45:332
364899271,12cyclictest0-21swapper/209:47:132
364899271,11cyclictest0-21swapper/208:09:412
364899271,10cyclictest0-21swapper/211:36:372
364899271,10cyclictest0-21swapper/211:36:372
364899271,10cyclictest0-21swapper/208:50:062
364899270,1cyclictest0-21swapper/207:34:212
364899270,16cyclictest0-21swapper/210:02:012
364899270,15cyclictest0-21swapper/209:55:132
364899270,14cyclictest0-21swapper/211:44:572
364899270,12cyclictest0-21swapper/211:00:172
364899270,12cyclictest0-21swapper/210:08:092
364899270,11cyclictest0-21swapper/211:26:212
364899270,11cyclictest0-21swapper/207:39:332
363699279,16cyclictest24702-21munin-run09:10:000
363699278,11cyclictest0-21swapper/010:35:130
363699272,12cyclictest12746-21idleruntime-cro11:10:010
363699271,14cyclictest3328-21munin-run09:35:020
364899267,6cyclictest26237-21hwlatdetect10:25:192
364899261,10cyclictest11888-21if_enp2s007:25:172
364899261,10cyclictest0-21swapper/208:00:252
3648992610,7cyclictest10964-21irqstats09:50:172
364899260,12cyclictest0-21swapper/209:18:252
364899260,12cyclictest0-21swapper/209:18:252
364899260,11cyclictest0-21swapper/209:20:212
364899260,10cyclictest0-21swapper/211:05:292
364399266,12cyclictest0-21swapper/111:55:261
363699269,15cyclictest6166-21perf10:55:010
363699261,9cyclictest9828-21kworker/0:109:49:320
363699261,9cyclictest4564-21kworker/0:012:05:110
364899251,9cyclictest0-21swapper/208:30:032
364899251,8cyclictest0-21swapper/211:16:132
364899251,16cyclictest0-21swapper/210:44:262
364899251,11cyclictest0-21swapper/209:35:452
364899251,11cyclictest0-21swapper/209:01:162
364899251,10cyclictest0-21swapper/208:19:532
364899251,10cyclictest0-21swapper/207:12:392
364899250,13cyclictest0-21swapper/212:10:572
364899250,11cyclictest0-21swapper/212:01:012
364899250,10cyclictest0-21swapper/212:06:012
364399252,21cyclictest0-21swapper/112:02:541
364399251,9cyclictest0-21swapper/112:29:061
363699252,9cyclictest8370-21kworker/0:108:31:390
363699251,10cyclictest20172-21kworker/0:209:35:160
3636992510,14cyclictest0-21swapper/007:30:010
364899241,11cyclictest0-21swapper/211:54:592
364899241,10cyclictest0-21swapper/208:21:312
3648992410,3cyclictest0-21swapper/212:33:222
364899240,9cyclictest0-21swapper/210:21:132
364899240,8cyclictest0-21swapper/212:23:252
364899240,16cyclictest0-21swapper/211:12:292
364899240,10cyclictest0-21swapper/212:25:062
3643992417,4cyclictest18373-21sed07:40:161
364399241,3cyclictest0-21swapper/111:38:411
364399241,3cyclictest0-21swapper/111:38:411
363699246,3cyclictest26887-21kworker/0:109:15:150
363699246,3cyclictest26887-21kworker/0:109:15:150
363699246,16cyclictest14325-21munin-run10:00:010
3636992413,2cyclictest5626-21kworker/0:007:18:010
363699240,4cyclictest0-21swapper/012:20:020
365299235,11cyclictest910-21systemd-logind08:56:243
364899239,3cyclictest0-21swapper/207:23:242
364899231,8cyclictest0-21swapper/210:19:092
364899230,8cyclictest0-21swapper/211:48:132
364899230,10cyclictest0-21swapper/210:55:222
364399233,16cyclictest89050irq/133-enp2s0-12:33:401
3643992315,3cyclictest0-21swapper/111:08:531
364399231,20cyclictest0-21swapper/111:13:521
364399231,19cyclictest12283-21perl08:40:151
364399231,18cyclictest2352-21grep10:45:131
364399231,13cyclictest13938-21users09:55:281
364399231,11cyclictest0-21swapper/110:13:031
364399231,11cyclictest0-21swapper/109:34:271
363699235,4cyclictest19376-21kworker/0:107:49:350
3636992316,2cyclictest32323-21kworker/0:111:58:240
363699231,11cyclictest13312-21kworker/0:207:38:190
365299222,13cyclictest0-21swapper/310:49:163
365299221,9cyclictest0-21swapper/308:12:473
365299221,6cyclictest0-21swapper/311:31:443
365299221,5cyclictest0-21swapper/309:42:203
364899228,9cyclictest0-21swapper/212:36:342
364899228,3cyclictest0-21swapper/209:09:402
364899228,3cyclictest0-21swapper/208:27:092
364899227,3cyclictest0-21swapper/208:37:272
364899221,10cyclictest0-21swapper/207:55:292
364899220,12cyclictest0-21swapper/210:52:062
364399227,7cyclictest0-21swapper/108:00:011
364399227,4cyclictest0-21swapper/110:21:451
364399225,2cyclictest7115-21munin-run08:30:021
3643992217,3cyclictest27519-21munin-run10:30:021
364399221,3cyclictest31301-21basename09:25:011
364399221,13cyclictest12245-21grep12:20:151
364399221,11cyclictest0-21swapper/112:11:571
364399221,11cyclictest0-21swapper/111:43:301
364399221,11cyclictest0-21swapper/111:26:181
364399221,11cyclictest0-21swapper/110:44:191
364399221,11cyclictest0-21swapper/110:19:581
364399221,10cyclictest0-21swapper/109:17:241
364399221,10cyclictest0-21swapper/109:17:241
363699228,3cyclictest26141-21kworker/0:208:12:080
363699227,2cyclictest19376-21kworker/0:107:51:000
363699221,4cyclictest0-21swapper/011:44:560
365299216,6cyclictest0-21swapper/309:22:463
365299216,5cyclictest0-21swapper/312:28:413
365299211,6cyclictest0-21swapper/311:39:023
365299211,6cyclictest0-21swapper/311:39:023
365299211,6cyclictest0-21swapper/311:03:123
364899210,9cyclictest0-21swapper/210:30:222
3643992116,2cyclictest10346-21interrupts08:35:201
364399211,3cyclictest108612sleep107:24:441
364399211,2cyclictest24698-21diskmemload11:30:111
364399211,15cyclictest0-21swapper/111:04:311
364399211,13cyclictest4526-21diskstats10:50:131
364399211,12cyclictest5625-21irqstats12:05:191
364399211,12cyclictest19129-21awk12:35:231
364399211,11cyclictest0-21swapper/109:53:341
363699216,2cyclictest13390-21kworker/0:008:47:230
363699215,8cyclictest25162-21kworker/0:010:30:010
363699212,3cyclictest18095-21munin-run08:55:000
363699212,11cyclictest652-21/usr/sbin/munin08:15:220
363699211,7cyclictest0-21swapper/012:24:180
363699211,5cyclictest0-21swapper/010:30:570
3636992114,3cyclictest8072-21kworker/0:011:15:150
363699211,3cyclictest1114-21munin-run09:30:020
363699211,12cyclictest22915-21kworker/0:110:23:390
220452210,4sleep1364399cyclictest07:50:011
365299209,3cyclictest31518-21/usr/sbin/munin09:25:183
365299208,4cyclictest0-21swapper/310:22:393
365299207,11cyclictest0-21swapper/307:25:213
365299205,5cyclictest0-21swapper/312:37:173
365299203,3cyclictest0-21swapper/312:03:283
365299202,10cyclictest9202-21/usr/sbin/munin12:15:133
365299202,10cyclictest7956-21/usr/sbin/munin09:45:243
365299201,13cyclictest0-21swapper/311:55:223
365299201,11cyclictest30236-21kworker/u8:312:30:123
365299201,11cyclictest2938-21/usr/sbin/munin08:20:183
365299201,11cyclictest19282-21sshd11:24:263
365299201,11cyclictest10758-21sshd07:23:263
365299201,10cyclictest4552-21grep09:35:203
365299201,10cyclictest31549-21strings11:50:203
365299201,10cyclictest0-21swapper/311:25:213
365299201,10cyclictest0-21swapper/308:53:313
365299200,7cyclictest0-21swapper/311:40:353
365299200,10cyclictest0-21swapper/309:13:003
364899200,10cyclictest0-21swapper/209:10:062
364399202,16cyclictest0-21swapper/109:35:351
364399202,16cyclictest0-21swapper/109:26:141
364399202,16cyclictest0-21swapper/107:28:321
364399202,16cyclictest0-21swapper/107:15:081
364399201,9cyclictest0-21swapper/110:34:351
3643992016,3cyclictest0-21swapper/109:00:521
3643992016,3cyclictest0-21swapper/108:21:551
364399201,3cyclictest24698-21diskmemload10:02:581
364399201,18cyclictest0-21swapper/110:05:181
364399201,11cyclictest9713-21cut12:15:151
364399201,11cyclictest31906-21latency_hist10:40:011
364399201,11cyclictest0-21swapper/111:15:111
364399201,10cyclictest8050-21cpuspeed_turbos09:45:111
364399201,10cyclictest24695-21latency_hist09:10:001
364399201,10cyclictest0-21swapper/111:52:231
364399201,10cyclictest0-21swapper/111:20:241
363699207,3cyclictest21325-21kworker/0:211:33:160
363699207,2cyclictest29606-21kworker/0:211:00:590
363699202,4cyclictest13312-21kworker/0:207:31:470
363699201,11cyclictest18321-21/usr/sbin/munin08:55:170
3636992010,3cyclictest909-21sssd_nss12:34:250
3636992010,3cyclictest87550irq/128-enp1s0-12:26:180
365299198,3cyclictest0-21swapper/310:01:283
365299196,7cyclictest0-21swapper/310:58:423
365299195,3cyclictest287862cat08:05:143
365299194,6cyclictest0-21swapper/310:39:273
365299193,2cyclictest87350irq/126-enp1s0-11:16:233
365299193,10cyclictest0-21swapper/311:10:393
365299192,13cyclictest23722-21needreboot09:05:203
365299191,16cyclictest20300-21timerandwakeup10:10:263
365299191,10cyclictest0-21swapper/312:06:473
364899194,7cyclictest87550irq/128-enp1s0-09:43:012
364399199,2cyclictest2543-21chrt08:17:591
364399192,16cyclictest0-21swapper/108:57:181
364399191,2cyclictest0-21swapper/107:53:491
364399191,17cyclictest0-21swapper/108:07:511
364399191,17cyclictest0-21swapper/107:38:171
364399191,16cyclictest0-21swapper/107:13:111
364399191,10cyclictest28034-21unixbench_multi08:00:251
363699198,2cyclictest20656-21kworker/0:310:17:350
363699196,3cyclictest3153-21kworker/0:009:40:320
363699196,3cyclictest26141-21kworker/0:207:59:320
363699194,3cyclictest18476-21kworker/0:210:10:190
363699191,3cyclictest0-21swapper/011:38:540
363699191,3cyclictest0-21swapper/011:38:540
363699191,15cyclictest11266-21pmu-power09:50:230
363699191,10cyclictest17988-21/usr/sbin/munin12:35:250
45562181,2sleep00-21swapper/012:04:480
365299186,3cyclictest0-21swapper/310:34:203
365299181,9cyclictest24301-21/usr/sbin/munin07:55:203
365299181,9cyclictest10605-21ntp_states08:35:233
365299181,9cyclictest0-21swapper/308:25:233
365299181,5cyclictest0-21swapper/311:06:553
365299181,4cyclictest12350-21/usr/sbin/munin09:55:143
365299181,15cyclictest0-21swapper/307:48:173
365299181,14cyclictest2009-21grep09:30:153
365299181,11cyclictest0-21swapper/310:15:433
365299181,11cyclictest0-21swapper/310:08:223
365299181,10cyclictest0-21swapper/312:23:523
365299181,10cyclictest0-21swapper/312:10:103
365299181,10cyclictest0-21swapper/310:28:373
364399181,16cyclictest0-21swapper/108:50:091
364399181,16cyclictest0-21swapper/108:46:431
364399181,16cyclictest0-21swapper/108:12:361
364399181,10cyclictest0-21swapper/108:30:141
363699187,3cyclictest31845-21ls11:50:260
363699185,4cyclictest11185-21kworker/0:208:40:150
363699185,2cyclictest87450irq/127-enp1s0-11:49:480
363699183,7cyclictest16758-21/usr/sbin/munin10:05:200
363699181,8cyclictest0-21swapper/007:13:260
363699181,3cyclictest26913-21awk09:15:010
3636991812,3cyclictest28710-21/usr/sbin/munin08:05:150
363699181,16cyclictest0-21swapper/010:55:560
284522180,3sleep30-21swapper/308:04:493
365299176,2cyclictest16507-21needreboot07:35:193
365299173,4cyclictest31712-21kworker/u8:110:50:113
365299171,8cyclictest20857-21runrttasks09:00:133
365299171,7cyclictest23448-21awk07:50:263
365299171,6cyclictest7990-21smart_sda07:15:233
365299171,6cyclictest0-21swapper/309:15:043
365299171,6cyclictest0-21swapper/309:15:043
365299171,6cyclictest0-21swapper/307:43:263
3652991712,3cyclictest11585-21timerwakeupswit09:50:243
365299171,15cyclictest0-21swapper/308:45:473
365299171,15cyclictest0-21swapper/307:13:043
365299171,10cyclictest0-21swapper/311:48:383
3643991714,2cyclictest0-21swapper/107:31:321
363699173,8cyclictest87550irq/128-enp1s0-08:20:530
363699173,2cyclictest87550irq/128-enp1s0-08:35:390
363699171,9cyclictest0-21swapper/008:02:250
363699171,6cyclictest29804-21fschecks_time09:20:150
363699171,6cyclictest17711-21/usr/sbin/munin07:40:170
363699171,6cyclictest0-21swapper/011:10:020
363699171,4cyclictest1973-21/usr/sbin/munin10:45:220
363699171,3cyclictest0-21swapper/010:03:220
363699171,13cyclictest18014-21sed11:20:160
365299161,14cyclictest0-21swapper/308:42:503
3652991611,3cyclictest13319-21/usr/sbin/munin07:30:263
363699163,4cyclictest24698-21diskmemload12:10:120
365299151,5cyclictest0-21swapper/308:19:253
3652991510,3cyclictest7342-21/usr/sbin/munin08:30:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional