You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 19:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu May 28, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
600023726,0sleep30-21swapper/306:49:373
701323626,0sleep20-21swapper/206:52:252
697023423,0sleep00-21swapper/006:51:550
711122622,0sleep10-21swapper/106:53:321
72519966,0cyclictest5463-21diskmemload09:29:471
72519960,0cyclictest0-21swapper/109:55:191
72509960,0cyclictest0-21swapper/011:58:280
35160,0ktimersoftd/30-21swapper/309:31:533
339960,0migration/328197-21sh11:08:573
339960,0migration/327182-21sh10:44:533
339960,0migration/326879-21sh11:25:543
339960,0migration/32397-21sh11:09:543
339960,0migration/322776-21aten_r1power_cu07:54:293
339960,0migration/314121-21aten_r1power_cu06:59:313
339960,0migration/311254-21sh10:34:533
29725060,0irq/27-eth0-tx-28399-21munin-node07:34:360
29725060,0irq/27-eth0-tx-27390-21munin-node07:09:350
29725060,0irq/27-eth0-tx-18013-21munin-node08:39:320
29715060,0irq/26-eth0-rx-9556-21ssh10:54:363
29715060,0irq/26-eth0-rx-9556-21ssh10:54:363
29715060,0irq/26-eth0-rx-6270-21tr09:34:313
29715060,0irq/26-eth0-rx-18279-21ssh12:01:243
259960,0migration/227022-21sh11:47:192
19160,0ktimersoftd/10-21swapper/107:24:361
149960,0migration/09428-21sh12:14:520
149960,0migration/03686-21taskset10:31:260
149960,0migration/026732-21sh11:25:260
149960,0migration/021294-21sh09:02:500
149960,0migration/020096-21sh10:00:370
149960,0migration/011897-21sh11:18:520
8215050,0irq/24-0000:00:5672-21cat08:54:202
72529955,0cyclictest0-21swapper/211:49:392
72529950,0cyclictest0-21swapper/210:59:592
72519955,0cyclictest0-21swapper/110:05:431
72519955,0cyclictest0-21swapper/109:44:361
72519955,0cyclictest0-21swapper/107:19:331
72519954,0cyclictest5954-21awk10:34:331
72519950,0cyclictest0-21swapper/112:22:441
72519950,0cyclictest0-21swapper/111:48:211
72519950,0cyclictest0-21swapper/111:29:361
72519950,0cyclictest0-21swapper/111:26:081
72519950,0cyclictest0-21swapper/110:25:161
72509955,0cyclictest0-21swapper/011:04:420
72509955,0cyclictest0-21swapper/006:59:330
72509954,0cyclictest3514-21snmpd09:33:200
72509953,0cyclictest593-21kworker/0:209:29:190
72509950,0cyclictest0-21swapper/009:40:500
339950,0migration/39164-21munin-node11:54:373
339950,0migration/3855-21latency11:49:363
339950,0migration/37596-21kthreadcore08:29:383
339950,0migration/36106-21kthreadcore07:39:373
339950,0migration/330801-21fschecks_count09:09:313
339950,0migration/327536-21aten_r1power_vo07:09:313
339950,0migration/326708-21latency10:24:363
339950,0migration/326512-21kthreadcore08:19:383
339950,0migration/324100-21kthreadcore09:24:363
339950,0migration/319903-21timerandwakeup08:59:423
339950,0migration/319830-21latency_hist11:44:183
339950,0migration/319763-21kthreadcore08:14:363
339950,0migration/318798-21hwlatdetect08:39:333
339950,0migration/317585-21sh12:20:233
339950,0migration/317226-21kthreadcore09:39:363
339950,0migration/314531-21kthreadcore11:19:353
339950,0migration/31306-21kthreadcore10:09:343
339950,0migration/311753-21diskmemload10:15:363
29725050,0irq/27-eth0-tx-523-21kthreadcore10:09:342
29725050,0irq/27-eth0-tx-4954-21munin-node10:34:322
29725050,0irq/27-eth0-tx-26045-21grep09:44:362
29725050,0irq/27-eth0-tx-18564-21ssh11:40:552
29725050,0irq/27-eth0-tx-11595-21telnet11:59:292
29725050,0irq/27-eth0-tx-0-21swapper/212:12:252
29725050,0irq/27-eth0-tx-0-21swapper/211:33:142
29725050,0irq/27-eth0-tx-0-21swapper/211:28:352
29725050,0irq/27-eth0-tx-0-21swapper/210:44:322
29725050,0irq/27-eth0-tx-0-21swapper/210:21:102
29725050,0irq/27-eth0-tx-0-21swapper/210:08:362
29725050,0irq/27-eth0-tx-0-21swapper/007:39:310
29725050,0irq/27-eth0-tx-0-21swapper/006:56:480
29715050,0irq/26-eth0-rx-6506-21df09:14:323
29715050,0irq/26-eth0-rx-16526-21kthreadcore10:59:373
29715050,0irq/26-eth0-rx-12516-21ssh09:56:433
29715050,0irq/26-eth0-rx-1139-21kthreadcore09:49:353
29715050,0irq/26-eth0-rx-0-21swapper/312:16:593
29715050,0irq/26-eth0-rx-0-21swapper/312:09:333
29715050,0irq/26-eth0-rx-0-21swapper/311:45:093
29715050,0irq/26-eth0-rx-0-21swapper/311:14:343
29715050,0irq/26-eth0-rx-0-21swapper/310:29:383
29715050,0irq/26-eth0-rx-0-21swapper/310:22:253
29715050,0irq/26-eth0-rx-0-21swapper/310:04:383
29715050,0irq/26-eth0-rx-0-21swapper/309:47:273
29715050,0irq/26-eth0-rx-0-21swapper/309:20:323
29715050,0irq/26-eth0-rx-0-21swapper/309:04:323
29715050,0irq/26-eth0-rx-0-21swapper/308:55:513
29715050,0irq/26-eth0-rx-0-21swapper/308:24:353
29715050,0irq/26-eth0-rx-0-21swapper/307:29:373
259950,0migration/24588-21cpuspeed_turbos11:14:302
259950,0migration/24517-21sh09:10:572
259950,0migration/230811-21kthreadcore10:49:352
259950,0migration/230811-21kthreadcore10:49:352
259950,0migration/227377-21taskset07:29:402
259950,0migration/226736-21munin-node07:04:442
259950,0migration/219768-21munin-node09:19:372
259950,0migration/217105-21sh08:59:372
259950,0migration/210574-21sh11:57:142
249950,0watchdog/23252-21sort09:29:382
249950,0watchdog/22952-21unixbench_multi10:29:412
249950,0watchdog/212214-21ssh09:55:502
19150,0ktimersoftd/10-21swapper/110:57:451
19150,0ktimersoftd/10-21swapper/110:57:451
179950,0migration/14642-21ksmtuned10:13:181
179950,0migration/12997-21cpuspeed_turbos07:39:321
179950,0migration/129634-21taskset09:28:041
179950,0migration/128751-21kthreadcore07:09:351
179950,0migration/127923-21sendmail_mailst09:04:411
179950,0migration/12455-21munin-node10:29:381
179950,0migration/111296-21kthreadcore06:54:371
169950,0watchdog/11517-21grep11:49:411
169950,0watchdog/113959-21if_eth010:39:331
149950,0migration/08435-21runrttasks09:14:360
149950,0migration/03356-21kthreadcore08:49:360
149950,0migration/030170-21latency_hist09:09:200
149950,0migration/028661-21taskset10:28:110
149950,0migration/027835-21cpuspeed_turbos12:09:310
149950,0migration/027524-21switchtime10:04:430
149950,0migration/027480-21kthreadcore09:04:380
149950,0migration/026567-21kthreadcore09:44:370
149950,0migration/024800-21kthreadcore07:04:380
149950,0migration/023254-21kthreadcore11:44:350
149950,0migration/018795-21sh12:02:540
149950,0migration/015713-21kthreadcore08:34:370
149950,0migration/015620-21kthreadcore10:19:340
149950,0migration/012121-21sh10:37:260
149950,0migration/012018-21kthreadcore08:09:360
72529944,0cyclictest3556-21lldpd11:04:222
72529943,0cyclictest9241-21awk07:19:322
72529943,0cyclictest5553-21awk10:54:342
72529943,0cyclictest5553-21awk10:54:342
72529943,0cyclictest3556-21lldpd12:14:252
72529943,0cyclictest3514-21snmpd08:46:342
72529943,0cyclictest3514-21snmpd08:41:042
72529943,0cyclictest3514-21snmpd08:33:042
72529943,0cyclictest3514-21snmpd08:25:342
72529943,0cyclictest3514-21snmpd07:56:042
72529943,0cyclictest3514-21snmpd07:25:042
72529943,0cyclictest20435-21awk12:04:332
72529943,0cyclictest17964-21awk08:14:342
72529943,0cyclictest17026-21awk07:49:342
72529943,0cyclictest16851-21sadc08:14:202
72529943,0cyclictest12108-21awk12:19:332
72529943,0cyclictest10245-21awk07:44:342
72519944,0cyclictest0-21swapper/109:18:071
72519943,0cyclictest6180-21awk10:14:341
72519943,0cyclictest3514-21snmpd09:51:201
72519943,0cyclictest3514-21snmpd08:48:341
72519943,0cyclictest3514-21snmpd07:29:491
72519943,0cyclictest3514-21snmpd07:18:341
72519943,0cyclictest32240-21awk08:49:331
72519943,0cyclictest21463-21awk11:04:341
72519943,0cyclictest15303-21awk09:19:331
72519943,0cyclictest14234-21awk10:19:331
72509943,0cyclictest6539-21awk09:54:320
72509943,0cyclictest6247-21awk10:14:340
72509943,0cyclictest593-21kworker/0:212:20:260
72509943,0cyclictest593-21kworker/0:210:52:560
72509943,0cyclictest593-21kworker/0:210:52:560
72509943,0cyclictest593-21kworker/0:210:11:590
72509943,0cyclictest5571-21awk08:29:360
72509943,0cyclictest4914-21awk11:34:330
72509943,0cyclictest3241-21irqbalance11:20:470
72509943,0cyclictest29182-21awk11:29:310
72509943,0cyclictest13494-21awk10:59:340
4140,0ktimersoftd/00-21swapper/011:44:070
339940,0migration/33770-21kthreadcore07:14:363
339940,0migration/332746-21kthreadcore07:59:363
339940,0migration/319039-21unixbench_multi10:39:423
29725040,0irq/27-eth0-tx-28577-21ssh11:09:242
29725040,0irq/27-eth0-tx-0-21swapper/211:19:472
29725040,0irq/27-eth0-tx-0-21swapper/210:26:082
29725040,0irq/27-eth0-tx-0-21swapper/209:59:212
29725040,0irq/27-eth0-tx-0-21swapper/209:49:352
29725040,0irq/27-eth0-tx-0-21swapper/209:39:552
29725040,0irq/27-eth0-tx-0-21swapper/209:34:272
29725040,0irq/27-eth0-tx-0-21swapper/209:24:212
29725040,0irq/27-eth0-tx-0-21swapper/209:14:312
29725040,0irq/27-eth0-tx-0-21swapper/209:04:312
29725040,0irq/27-eth0-tx-0-21swapper/008:54:350
29725040,0irq/27-eth0-tx-0-21swapper/007:24:440
29715040,0irq/26-eth0-rx-0-21swapper/312:04:273
29715040,0irq/26-eth0-rx-0-21swapper/311:34:423
29715040,0irq/26-eth0-rx-0-21swapper/311:29:343
29715040,0irq/26-eth0-rx-0-21swapper/310:49:313
29715040,0irq/26-eth0-rx-0-21swapper/310:49:313
29715040,0irq/26-eth0-rx-0-21swapper/310:00:413
259940,0migration/28398-21kthreadcore10:14:362
259940,0migration/27256-21kthreadcore11:34:362
259940,0migration/231527-21kthreadcore07:09:362
259940,0migration/22736-21aten2.4-expect07:39:312
259940,0migration/217901-21kthreadcore06:59:372
259940,0migration/214533-21kthreadcore10:39:352
249940,0watchdog/20-21swapper/208:19:382
179940,0migration/17550-21kthreadcore11:34:361
179940,0migration/17121-21kthreadcore08:04:371
179940,0migration/131369-21kthreadcore11:09:351
179940,0migration/129183-21kthreadcore12:09:361
179940,0migration/126335-21latency10:44:361
179940,0migration/123357-21taskset08:17:311
179940,0migration/120766-21irqrtprio12:04:341
179940,0migration/118265-21cpuspeed_turbos08:39:311
179940,0migration/116056-21kthreadcore11:59:351
179940,0migration/114226-21fschecks_count09:59:321
179940,0migration/112989-21taskset08:57:361
169940,0watchdog/19993-21ssh12:16:331
169940,0watchdog/17792-21kthreadcore08:29:381
169940,0watchdog/15463-21diskmemload11:40:011
169940,0watchdog/129023-21munin-node10:49:291
169940,0watchdog/129023-21munin-node10:49:291
169940,0watchdog/124391-21perl08:19:341
169940,0watchdog/11844-21timerwakeupswit07:34:421
169940,0watchdog/118366-21python10:59:411
169940,0watchdog/116193-21sed08:09:421
169940,0watchdog/112612-21ssh09:35:541
169940,0watchdog/112215-21if_err_eth1.208:34:341
169940,0watchdog/110921-21ssh11:16:011
169940,0watchdog/10-21swapper/111:54:371
169940,0watchdog/10-21swapper/109:09:301
149940,0migration/031807-21kthreadcore11:09:350
149940,0migration/031258-21latency_hist08:49:200
149940,0migration/026461-21runrttasks12:06:550
149940,0migration/025738-21kthreadcore07:29:370
149940,0migration/015406-21irqcore09:19:340
72529933,0cyclictest0-21swapper/207:14:422
72529932,0cyclictest3514-21snmpd08:06:042
72529932,0cyclictest3514-21snmpd08:01:192
72529932,0cyclictest32288-21awk08:49:332
72529932,0cyclictest29701-21kthreadcore07:34:372
72519932,0cyclictest3514-21snmpd08:27:021
72519932,0cyclictest30329-21awk07:59:341
72519932,0cyclictest23728-21awk07:54:341
72519932,0cyclictest19741-21sensors08:59:411
72519930,0cyclictest24161-21grep07:04:381
72509933,0cyclictest593-21kworker/0:211:49:200
72509933,0cyclictest593-21kworker/0:210:54:200
72509933,0cyclictest593-21kworker/0:210:54:200
72509933,0cyclictest593-21kworker/0:210:44:290
72509933,0cyclictest593-21kworker/0:210:39:210
72509933,0cyclictest593-21kworker/0:209:49:230
72509933,0cyclictest593-21kworker/0:209:34:230
72509933,0cyclictest593-21kworker/0:208:24:210
72509933,0cyclictest593-21kworker/0:208:19:270
72509933,0cyclictest593-21kworker/0:207:54:330
72509933,0cyclictest593-21kworker/0:207:49:310
72509933,0cyclictest593-21kworker/0:207:44:230
72509933,0cyclictest593-21kworker/0:207:19:210
72509933,0cyclictest593-21kworker/0:207:14:430
29725030,0irq/27-eth0-tx-29601-21aten2.4-expect07:59:310
29725030,0irq/27-eth0-tx-0-21swapper/008:14:230
29725030,0irq/27-eth0-tx-0-21swapper/008:04:310
29715030,0irq/26-eth0-rx-0-21swapper/308:49:303
29715030,0irq/26-eth0-rx-0-21swapper/308:44:203
29715030,0irq/26-eth0-rx-0-21swapper/308:13:453
29715030,0irq/26-eth0-rx-0-21swapper/308:05:003
29715030,0irq/26-eth0-rx-0-21swapper/307:49:303
29715030,0irq/26-eth0-rx-0-21swapper/307:44:393
29715030,0irq/26-eth0-rx-0-21swapper/307:34:303
29715030,0irq/26-eth0-rx-0-21swapper/307:24:313
29715030,0irq/26-eth0-rx-0-21swapper/307:19:293
29715030,0irq/26-eth0-rx-0-21swapper/307:04:343
29715030,0irq/26-eth0-rx-0-21swapper/306:54:313
249930,0watchdog/20-21swapper/208:34:342
249930,0watchdog/20-21swapper/206:54:342
169930,0watchdog/15463-21diskmemload09:43:181
169930,0watchdog/116167-21aten2.4-expect07:49:301
169930,0watchdog/10-21swapper/111:19:291
169930,0watchdog/10-21swapper/106:59:341
29715020,0irq/26-eth0-rx-0-21swapper/308:34:353
169920,0watchdog/10-21swapper/107:44:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional