You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 06:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Apr 25, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
245225342,0sleep00-21swapper/018:53:120
304450350,0irq/25-eth10-21swapper/318:53:013
280223425,0sleep20-21swapper/218:57:112
280122824,0sleep10-21swapper/118:57:101
8215060,0irq/24-0000:00:17752-21latency_hist21:27:292
339960,0migration/38572-21sh21:00:553
339960,0migration/3572-21sh21:35:203
339960,0migration/323978-21sh21:08:193
339960,0migration/317990-21sh21:27:343
339960,0migration/315968-21sh21:03:473
339960,0migration/313441-21munin-node00:22:513
29725060,0irq/27-eth0-tx-13672-21munin-node20:42:530
29725060,0irq/27-eth0-tx-12719-21munin-node20:17:510
29715060,0irq/26-eth0-rx-3862-21kthreadcore22:17:473
29715060,0irq/26-eth0-rx-23420-21sh23:09:373
29715060,0irq/26-eth0-rx-1723-21munin-node22:37:443
29715060,0irq/26-eth0-rx-0-21swapper/319:12:413
29099960,0cyclictest24777-21tail23:12:412
29089966,0cyclictest0-21swapper/123:22:521
149960,0migration/08792-21aten_r1power_fr23:02:420
149960,0migration/023959-21sh23:32:040
149960,0migration/016407-21latency_hist19:07:310
4150,0ktimersoftd/00-21swapper/023:23:480
339950,0migration/39249-21kthreadcore20:37:463
339950,0migration/38082-21taskset21:18:183
339950,0migration/3456-21latency_hist22:57:293
339950,0migration/33304-21kthreadcore21:57:463
339950,0migration/330186-21munin-node23:32:533
339950,0migration/329506-21kthreadcore20:27:463
339950,0migration/326200-21munin-node20:02:473
339950,0migration/324006-21sh22:50:273
339950,0migration/322750-21sh23:48:583
339950,0migration/321040-21gltestperf20:47:433
339950,0migration/319943-21kthreadcore22:07:463
339950,0migration/319068-21kernelversion21:47:463
339950,0migration/317329-21aten_r1power_vo22:27:413
339950,0migration/315911-21kthreadcore20:42:473
339950,0migration/312696-21latency_hist20:17:303
29725050,0irq/27-eth0-tx-5330-21missed_timers23:57:472
29725050,0irq/27-eth0-tx-31778-21ssh22:54:512
29725050,0irq/27-eth0-tx-29762-21munin-node19:17:490
29725050,0irq/27-eth0-tx-28164-21kthreadcore00:12:472
29725050,0irq/27-eth0-tx-17190-21munin-node22:27:452
29725050,0irq/27-eth0-tx-12653-21ssh22:42:462
29725050,0irq/27-eth0-tx-0-21swapper/223:44:412
29725050,0irq/27-eth0-tx-0-21swapper/222:47:412
29725050,0irq/27-eth0-tx-0-21swapper/222:04:052
29725050,0irq/27-eth0-tx-0-21swapper/221:28:242
29725050,0irq/27-eth0-tx-0-21swapper/220:59:122
29715050,0irq/26-eth0-rx-9173-21iostat_ios23:42:463
29715050,0irq/26-eth0-rx-8752-21latency_hist22:42:293
29715050,0irq/26-eth0-rx-6559-21runrttasks23:17:513
29715050,0irq/26-eth0-rx-509-21grep23:57:423
29715050,0irq/26-eth0-rx-3025-21kthreadcore21:37:473
29715050,0irq/26-eth0-rx-28272-21kthreadcore20:52:453
29715050,0irq/26-eth0-rx-27680-21kthreadcore23:12:473
29715050,0irq/26-eth0-rx-25312-21aten_r1power_vo22:32:443
29715050,0irq/26-eth0-rx-25063-21idleruntime23:52:433
29715050,0irq/26-eth0-rx-23393-21ssh23:30:253
29715050,0irq/26-eth0-rx-18466-21cut00:07:463
29715050,0irq/26-eth0-rx-15760-21ssh21:22:513
29715050,0irq/26-eth0-rx-11157-21kthreadcore19:02:473
29715050,0irq/26-eth0-rx-0-21swapper/323:22:463
29715050,0irq/26-eth0-rx-0-21swapper/323:02:303
29715050,0irq/26-eth0-rx-0-21swapper/322:06:483
29715050,0irq/26-eth0-rx-0-21swapper/319:27:463
29715050,0irq/26-eth0-rx-0-21swapper/300:17:463
29715050,0irq/26-eth0-rx-0-21swapper/300:02:473
29099955,0cyclictest0-21swapper/222:35:002
29099950,0cyclictest986-21diskmemload23:51:372
29099950,0cyclictest0-21swapper/200:23:282
29089950,0cyclictest0-21swapper/121:26:211
29089950,0cyclictest0-21swapper/121:15:191
29079955,0cyclictest986-21diskmemload23:59:540
29079955,0cyclictest0-21swapper/022:32:050
29079953,0cyclictest1356-21kworker/0:123:38:510
29079950,0cyclictest0-21swapper/022:24:410
29079950,0cyclictest0-21swapper/021:38:010
29079950,0cyclictest0-21swapper/021:25:050
29079950,0cyclictest0-21swapper/000:19:370
29079950,0cyclictest0-21swapper/000:04:100
27150,0ktimersoftd/20-21swapper/223:23:482
259950,0migration/28558-21sh21:19:302
259950,0migration/231597-21latency_hist20:07:302
259950,0migration/230322-21fschecks_time19:17:432
259950,0migration/228962-21kthreadcore21:12:462
259950,0migration/226047-21sh21:32:392
259950,0migration/22019-21munin-node22:17:452
259950,0migration/216935-21cpuspeed_turbos23:07:422
259950,0migration/215543-21sh23:05:002
259950,0migration/213601-21latency_hist20:42:302
259950,0migration/210350-21fschecks_time21:42:432
19150,0ktimersoftd/10-21swapper/122:30:021
179950,0migration/17415-21latency_hist00:22:291
179950,0migration/126177-21kthreadcore00:12:471
179950,0migration/125974-21latency_hist21:32:301
179950,0migration/124525-21sh22:51:561
179950,0migration/123843-21kthreadcore20:22:471
149950,0migration/0987-21gltestperf20:32:450
149950,0migration/08641-21ntp_kernel_pll_18:57:490
149950,0migration/0850-21taskset20:56:220
149950,0migration/07809-21taskset22:40:120
149950,0migration/06727-21kthreadcore19:47:460
149950,0migration/05423-21kthreadcore23:17:460
149950,0migration/04634-21fschecks_time19:22:420
149950,0migration/04634-21fschecks_time19:22:420
149950,0migration/04204-21kthreadcore20:57:460
149950,0migration/03521-21kthreadcore21:57:460
149950,0migration/030066-21kthreadcore21:12:470
149950,0migration/029520-21kthreadcore20:27:460
149950,0migration/028291-21kthreadcore21:32:480
149950,0migration/027914-21kthreadcore23:32:470
149950,0migration/027463-21kthreadcore21:52:460
149950,0migration/026756-21kthreadcore23:12:460
149950,0migration/025717-21fschecks_count22:32:450
149950,0migration/025626-21kthreadcore19:37:460
149950,0migration/025350-21sh21:31:170
149950,0migration/023149-21aten_r1power_cu19:12:410
149950,0migration/023030-21taskset23:08:290
149950,0migration/02196-21kthreadcore19:42:470
149950,0migration/019771-21kthreadcore23:47:450
149950,0migration/019606-21kthreadcore21:47:470
149950,0migration/018246-21hwlatdetect22:07:430
149950,0migration/017934-21fschecks_count19:32:440
149950,0migration/016624-21sh22:05:550
149950,0migration/016218-21munin-node19:27:490
149950,0migration/012646-21munin-node19:52:450
8215040,0irq/24-0000:00:0-21swapper/223:17:292
8215040,0irq/24-0000:00:0-21swapper/220:52:302
8215040,0irq/24-0000:00:0-21swapper/220:22:252
339940,0migration/35483-21kthreadcore19:22:453
339940,0migration/35483-21kthreadcore19:22:453
339940,0migration/332665-21kthreadcore19:42:463
339940,0migration/329102-21kthreadcore00:12:483
339940,0migration/32574-21kthreadcore23:37:463
339940,0migration/324434-21kthreadcore20:22:473
339940,0migration/319793-21kthreadcore19:57:473
339940,0migration/313907-21kthreadcore19:52:473
29725040,0irq/27-eth0-tx-379-21if_eth100:17:442
29725040,0irq/27-eth0-tx-25854-21ssh21:52:412
29725040,0irq/27-eth0-tx-17865-21ssh22:07:422
29725040,0irq/27-eth0-tx-0-21swapper/223:52:352
29725040,0irq/27-eth0-tx-0-21swapper/223:32:422
29725040,0irq/27-eth0-tx-0-21swapper/221:57:412
29725040,0irq/27-eth0-tx-0-21swapper/221:37:432
29725040,0irq/27-eth0-tx-0-21swapper/221:07:352
29725040,0irq/27-eth0-tx-0-21swapper/020:38:470
29725040,0irq/27-eth0-tx-0-21swapper/019:57:500
29725040,0irq/27-eth0-tx-0-21swapper/019:02:420
29715040,0irq/26-eth0-rx-0-21swapper/322:52:333
29715040,0irq/26-eth0-rx-0-21swapper/322:22:593
29715040,0irq/26-eth0-rx-0-21swapper/322:12:573
29715040,0irq/26-eth0-rx-0-21swapper/321:52:463
29715040,0irq/26-eth0-rx-0-21swapper/321:43:133
29715040,0irq/26-eth0-rx-0-21swapper/321:13:163
29099944,0cyclictest0-21swapper/200:07:332
29099943,0cyclictest3556-21lldpd23:37:482
29099943,0cyclictest3556-21lldpd22:58:172
29099943,0cyclictest3556-21lldpd20:29:172
29099943,0cyclictest3556-21lldpd19:51:172
29099943,0cyclictest3514-21snmpd19:08:382
29099943,0cyclictest21231-21awk20:47:442
29099943,0cyclictest18723-21awk21:47:452
29099940,0cyclictest0-21swapper/200:02:302
29089944,0cyclictest0-21swapper/122:54:381
29089944,0cyclictest0-21swapper/122:07:491
29089943,0cyclictest9030-21awk23:42:451
29089943,0cyclictest7040-21awk20:12:461
29089943,0cyclictest4148-21awk18:57:451
29089943,0cyclictest3556-21lldpd23:49:481
29089943,0cyclictest3556-21lldpd21:37:471
29089943,0cyclictest3556-21lldpd20:10:171
29089943,0cyclictest3556-21lldpd19:35:171
29089943,0cyclictest3556-21lldpd19:29:471
29089943,0cyclictest3514-21snmpd22:25:101
29089943,0cyclictest26856-21awk20:27:431
29089943,0cyclictest25554-21awk23:12:441
29089943,0cyclictest2255-21irqcore20:57:451
29089943,0cyclictest13697-21awk20:17:461
29089943,0cyclictest10391-21awk22:02:451
29079944,0cyclictest0-21swapper/022:47:420
29079944,0cyclictest0-21swapper/022:12:320
29079943,0cyclictest8998-21awk23:42:440
29079943,0cyclictest8478-21awk00:22:440
29079943,0cyclictest24797-21awk00:12:450
29079943,0cyclictest2042-21awk22:17:450
29079943,0cyclictest20296-21awk20:22:440
29079942,0cyclictest3556-21lldpd20:47:470
259940,0migration/26095-21kthreadcore19:22:462
259940,0migration/26095-21kthreadcore19:22:462
259940,0migration/24789-21kthreadcore18:57:462
259940,0migration/226152-21diskmemload22:12:432
259940,0migration/218798-21kthreadcore19:32:472
259940,0migration/21593-21kthreadcore20:32:482
259940,0migration/212381-21needreboot20:37:482
259940,0migration/212249-21kthreadcore19:27:462
259940,0migration/21039-21kthreadcore20:07:452
249940,0watchdog/26751-21timerandwakeup22:37:522
249940,0watchdog/2303062sleep219:39:562
249940,0watchdog/214861-21ssh21:02:482
249940,0watchdog/20-21swapper/223:27:322
249940,0watchdog/20-21swapper/222:22:482
179940,0migration/18561-21fschecks_time00:02:431
179940,0migration/16280-21irqrtprio19:47:461
179940,0migration/15238-21latency23:57:471
179940,0migration/14736-21fschecks_time19:22:421
179940,0migration/14736-21fschecks_time19:22:421
179940,0migration/13672-21kthreadcore22:37:461
179940,0migration/1353-21kthreadcore19:17:471
179940,0migration/13428-21kthreadcore23:17:451
179940,0migration/130907-21kthreadcore21:52:471
179940,0migration/130595-21missed_timers22:32:501
179940,0migration/12969-21kthreadcore22:17:461
179940,0migration/129233-21kthreadcore19:37:471
179940,0migration/128476-21needreboot19:12:491
179940,0migration/126365-21kthreadcore23:32:471
179940,0migration/126103-21fschecks_time22:12:431
179940,0migration/125881-21kthreadcore23:52:451
179940,0migration/125230-21taskset21:30:541
179940,0migration/122316-21kthreadcore21:47:471
179940,0migration/121717-21kthreadcore20:47:461
179940,0migration/120451-21kthreadcore00:07:471
179940,0migration/119121-21kthreadcore23:27:471
179940,0migration/114104-21fschecks_count20:42:431
179940,0migration/113017-21kthreadcore21:42:471
179940,0migration/112278-21kthreadcore00:22:461
169940,0watchdog/18743-21ssh21:20:001
169940,0watchdog/17798-21ssh21:58:281
169940,0watchdog/132427-21munin-node23:37:441
169940,0watchdog/121964-21kthreadcore23:07:481
169940,0watchdog/116567-21ssh21:05:201
169940,0watchdog/111533-21kthreadcore23:02:481
169940,0watchdog/10-21swapper/121:07:441
169940,0watchdog/10-21swapper/120:02:481
149940,0migration/026447-21kthreadcore20:02:470
149940,0migration/015628-21runrttasks22:44:300
149940,0migration/010732-21if_eth1.221:42:450
339930,0migration/316475-21runrttasks19:07:343
29715030,0irq/26-eth0-rx-0-21swapper/318:57:403
29099932,0cyclictest9853-21munin-node19:02:432
29099932,0cyclictest3556-21lldpd20:12:472
29099932,0cyclictest31930-21cut19:42:462
29099932,0cyclictest20270-21grep19:57:472
29099932,0cyclictest12574-21basename19:52:442
29089933,0cyclictest3556-21lldpd19:46:471
29089932,0cyclictest3556-21lldpd19:58:171
29089932,0cyclictest27949-21awk20:52:441
29089932,0cyclictest1438-21awk22:57:421
29089932,0cyclictest12693-21awk19:52:451
29089932,0cyclictest12444-21nfs_client20:37:481
29089932,0cyclictest10707-21awk19:02:451
29089931,0cyclictest16484-21munin-node19:07:471
29079933,0cyclictest14628-21kworker/0:022:57:410
29079933,0cyclictest14628-21kworker/0:022:53:090
29079933,0cyclictest14628-21kworker/0:021:07:410
29079933,0cyclictest14628-21kworker/0:021:03:170
29079933,0cyclictest14628-21kworker/0:020:13:230
29079933,0cyclictest14628-21kworker/0:020:10:210
29079933,0cyclictest1356-21kworker/0:123:53:510
29079933,0cyclictest1356-21kworker/0:100:10:230
29079932,0cyclictest2502-21awk21:17:450
249930,0watchdog/223984-21munin-node19:12:452
169930,0watchdog/10-21swapper/122:42:521
169930,0watchdog/10-21swapper/120:32:481
329920,0watchdog/35272-21tr20:32:493
29715020,0irq/26-eth0-rx-0-21swapper/320:12:463
29715020,0irq/26-eth0-rx-0-21swapper/320:07:383
29715020,0irq/26-eth0-rx-0-21swapper/319:47:413
29715020,0irq/26-eth0-rx-0-21swapper/319:37:483
29715020,0irq/26-eth0-rx-0-21swapper/319:32:353
29715020,0irq/26-eth0-rx-0-21swapper/319:17:473
29099920,0cyclictest19305-21date20:22:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional