You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 17:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Apr 23, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1594623626,0sleep30-21swapper/306:56:093
1582223524,0sleep00-21swapper/006:54:420
1575023425,0sleep20-21swapper/206:53:522
1580823324,0sleep10-21swapper/106:54:311
161469988,0cyclictest0-21swapper/008:57:570
35160,0ktimersoftd/30-21swapper/310:36:313
339960,0migration/35699-21sh09:31:243
339960,0migration/34938-21aten_r1power_cu07:37:493
339960,0migration/321955-21sh09:20:573
339960,0migration/319625-21sh11:18:433
339960,0migration/311045-21sh11:53:383
29725060,0irq/27-eth0-tx-30468-21munin-node09:47:512
29725060,0irq/27-eth0-tx-14241-21dump-pmu-power08:57:380
29715060,0irq/26-eth0-rx-4129-21sh09:08:013
29715060,0irq/26-eth0-rx-18920-21cut08:57:533
161489965,0cyclictest18052-21gzip12:17:582
161489960,0cyclictest0-21swapper/212:02:122
161469966,0cyclictest0-21swapper/012:18:150
161469960,0cyclictest0-21swapper/011:52:520
161469960,0cyclictest0-21swapper/009:45:000
161469960,0cyclictest0-21swapper/007:12:520
149960,0migration/011820-21sh12:17:100
8215050,0irq/24-0000:00:4699-21sh07:37:392
8215050,0irq/24-0000:00:14426-21cat09:17:382
339950,0migration/3692-21fschecks_time08:22:513
339950,0migration/331782-21munin-node08:18:003
339950,0migration/331451-21munin-node07:32:503
339950,0migration/32882-21kthreadcore09:47:563
339950,0migration/328210-21latency07:02:573
339950,0migration/322787-21irqrtprio11:02:543
339950,0migration/322551-21latency07:22:563
339950,0migration/321334-21fschecks_time12:02:523
339950,0migration/319633-21kthreadcore07:47:553
339950,0migration/319442-21munin-node10:37:573
339950,0migration/316846-21taskset07:19:593
339950,0migration/313546-21latency_hist08:32:393
339950,0migration/312432-21munin-node09:33:033
339950,0migration/312232-21fschecks_time07:42:523
339950,0migration/312086-21taskset08:04:033
29725050,0irq/27-eth0-tx-4924-21munin-node07:37:510
29725050,0irq/27-eth0-tx-3438-21ssh11:08:312
29725050,0irq/27-eth0-tx-30615-21munin-node07:32:530
29725050,0irq/27-eth0-tx-29264-21munin-node11:27:562
29725050,0irq/27-eth0-tx-28093-21pmu-power09:42:582
29725050,0irq/27-eth0-tx-0-21swapper/212:23:122
29725050,0irq/27-eth0-tx-0-21swapper/211:37:292
29725050,0irq/27-eth0-tx-0-21swapper/210:49:132
29725050,0irq/27-eth0-tx-0-21swapper/210:35:312
29725050,0irq/27-eth0-tx-0-21swapper/210:27:492
29725050,0irq/27-eth0-tx-0-21swapper/209:34:412
29725050,0irq/27-eth0-tx-0-21swapper/209:22:552
29725050,0irq/27-eth0-tx-0-21swapper/008:20:030
29725050,0irq/27-eth0-tx-0-21swapper/008:03:020
29715050,0irq/26-eth0-rx-6559-21runrttasks12:08:273
29715050,0irq/26-eth0-rx-4243-21ssh10:29:213
29715050,0irq/26-eth0-rx-30582-21df_abs10:07:503
29715050,0irq/26-eth0-rx-24107-21munin-node09:22:543
29715050,0irq/26-eth0-rx-10189-21tr10:52:563
29715050,0irq/26-eth0-rx-0-21swapper/312:26:343
29715050,0irq/26-eth0-rx-0-21swapper/312:18:093
29715050,0irq/26-eth0-rx-0-21swapper/311:59:073
29715050,0irq/26-eth0-rx-0-21swapper/311:42:453
29715050,0irq/26-eth0-rx-0-21swapper/311:32:553
29715050,0irq/26-eth0-rx-0-21swapper/311:07:553
29715050,0irq/26-eth0-rx-0-21swapper/310:22:533
29715050,0irq/26-eth0-rx-0-21swapper/310:20:503
29715050,0irq/26-eth0-rx-0-21swapper/310:12:553
29715050,0irq/26-eth0-rx-0-21swapper/309:43:183
29715050,0irq/26-eth0-rx-0-21swapper/309:37:523
27150,0ktimersoftd/20-21swapper/210:14:132
259950,0migration/27888-21kthreadcore12:12:552
259950,0migration/232255-21fschecks_time07:57:522
259950,0migration/224806-21kthreadcore11:02:552
259950,0migration/224362-21kthreadcore09:02:562
259950,0migration/222531-21cpuspeed_turbos10:02:502
259950,0migration/221796-21kthreadcore08:37:552
249950,0watchdog/228762-21ssh10:45:362
249950,0watchdog/213707-21unixbench_multi08:53:012
179950,0migration/17298-21kthreadcore11:32:551
179950,0migration/128163-21taskset11:25:151
179950,0migration/126885-21runrttasks12:24:251
179950,0migration/124881-21kthreadcore11:42:541
179950,0migration/11890-21kthreadcore07:07:561
179950,0migration/117914-21fschecks_time07:22:511
179950,0migration/113900-21fschecks_time11:17:501
161489955,0cyclictest0-21swapper/211:53:232
161489950,0cyclictest0-21swapper/211:50:422
161489950,0cyclictest0-21swapper/211:18:322
161489950,0cyclictest0-21swapper/208:59:192
161479954,0cyclictest3514-21snmpd10:46:591
161469955,0cyclictest0-21swapper/011:44:020
161469950,0cyclictest0-21swapper/011:41:210
161469950,0cyclictest0-21swapper/010:52:480
161469950,0cyclictest0-21swapper/010:03:550
161469950,0cyclictest0-21swapper/009:17:530
169950,0watchdog/13900-21munin-node10:08:001
169950,0watchdog/128195-21sh11:04:441
169950,0watchdog/112803-21unixbench_multi08:28:011
149950,0migration/08206-21kthreadcore10:32:570
149950,0migration/06825-21munin-node10:12:530
149950,0migration/05031-21munin-node07:58:010
149950,0migration/04848-21latency_hist11:32:370
149950,0migration/0415-21kthreadcore09:07:560
149950,0migration/0397-21kthreadcore10:27:540
149950,0migration/031889-21kthreadcore11:47:540
149950,0migration/029911-21fschecks_count11:07:510
149950,0migration/029741-21kthreadcore08:42:540
149950,0migration/029728-21taskset10:47:440
149950,0migration/029677-21munin-node11:27:500
149950,0migration/028891-21sh11:06:450
149950,0migration/026762-21unixbench_singl12:03:020
149950,0migration/02411-21kthreadcore09:27:560
149950,0migration/023418-21taskset07:24:170
149950,0migration/022362-21kthreadcore07:47:560
149950,0migration/022259-21kthreadcore12:22:540
149950,0migration/021170-21sh10:41:550
149950,0migration/019315-21kthreadcore06:57:540
149950,0migration/015752-21kthreadcore10:57:540
149950,0migration/014608-21kthreadcore07:17:580
339940,0migration/36644-21kthreadcore12:12:553
339940,0migration/329307-21cpuspeed_turbos11:47:493
339940,0migration/322288-21unixbench_multi06:58:003
339940,0migration/319944-21fschecks_time08:12:523
339940,0migration/317411-21kthreadcore09:57:553
29725040,0irq/27-eth0-tx-30436-21kthreadcore12:07:532
29725040,0irq/27-eth0-tx-18560-21ssh11:37:572
29725040,0irq/27-eth0-tx-12482-21ssh09:53:312
29725040,0irq/27-eth0-tx-0-21swapper/212:02:542
29725040,0irq/27-eth0-tx-0-21swapper/211:42:562
29725040,0irq/27-eth0-tx-0-21swapper/211:23:022
29725040,0irq/27-eth0-tx-0-21swapper/211:12:502
29725040,0irq/27-eth0-tx-0-21swapper/210:57:462
29725040,0irq/27-eth0-tx-0-21swapper/210:52:462
29725040,0irq/27-eth0-tx-0-21swapper/210:24:102
29725040,0irq/27-eth0-tx-0-21swapper/210:07:562
29725040,0irq/27-eth0-tx-0-21swapper/209:38:062
29725040,0irq/27-eth0-tx-0-21swapper/209:27:402
29725040,0irq/27-eth0-tx-0-21swapper/209:18:132
29725040,0irq/27-eth0-tx-0-21swapper/209:07:452
29725040,0irq/27-eth0-tx-0-21swapper/008:37:550
29725040,0irq/27-eth0-tx-0-21swapper/008:32:530
29725040,0irq/27-eth0-tx-0-21swapper/007:55:240
29725040,0irq/27-eth0-tx-0-21swapper/007:02:520
29715040,0irq/26-eth0-rx-13655-21ps08:07:543
29715040,0irq/26-eth0-rx-0-21swapper/311:38:303
29715040,0irq/26-eth0-rx-0-21swapper/311:28:113
29715040,0irq/26-eth0-rx-0-21swapper/311:22:433
29715040,0irq/26-eth0-rx-0-21swapper/311:15:033
29715040,0irq/26-eth0-rx-0-21swapper/310:57:483
29715040,0irq/26-eth0-rx-0-21swapper/310:47:483
29715040,0irq/26-eth0-rx-0-21swapper/310:43:463
29715040,0irq/26-eth0-rx-0-21swapper/310:02:463
29715040,0irq/26-eth0-rx-0-21swapper/309:52:403
29715040,0irq/26-eth0-rx-0-21swapper/309:14:133
29715040,0irq/26-eth0-rx-0-21swapper/309:02:583
29715040,0irq/26-eth0-rx-0-21swapper/308:38:573
29715040,0irq/26-eth0-rx-0-21swapper/308:27:493
29715040,0irq/26-eth0-rx-0-21swapper/307:52:553
29715040,0irq/26-eth0-rx-0-21swapper/307:07:523
259940,0migration/29895-21kthreadcore08:27:552
259940,0migration/28269-21kthreadcore08:02:572
259940,0migration/2571-21kthreadcore07:07:552
259940,0migration/228270-21load07:02:572
259940,0migration/227896-21irqcore08:42:532
259940,0migration/221066-21kthreadcore07:47:552
259940,0migration/214775-21kthreadcore07:42:562
259940,0migration/214257-21taskset09:57:482
249940,0watchdog/23964-21cut07:12:492
249940,0watchdog/213666-21ssh10:37:442
249940,0watchdog/20-21swapper/207:17:572
179940,0migration/1997-21kthreadcore09:27:551
179940,0migration/19162-21kthreadcore09:52:551
179940,0migration/15861-21forks10:52:511
179940,0migration/13737-21kthreadcore07:57:561
179940,0migration/132606-21kthreadcore11:47:541
179940,0migration/130625-21fschecks_time10:27:501
179940,0migration/126683-21hddtemp_smartct08:17:511
179940,0migration/124998-21switchtime08:13:001
179940,0migration/122011-21aten2.4-expect10:22:481
179940,0migration/121295-21kthreadcore06:57:551
179940,0migration/120129-21kthreadcore07:47:551
179940,0migration/119646-21kthreadcore09:37:561
179940,0migration/115105-21kthreadcore11:57:531
179940,0migration/114061-21kthreadcore12:17:541
179940,0migration/113774-21aten_r1power_fr10:37:481
179940,0migration/113485-21latency_hist08:32:391
179940,0migration/112559-21sh10:14:241
179940,0migration/112164-21kthreadcore07:17:571
161489943,0cyclictest3556-21lldpd10:19:032
161489943,0cyclictest3556-21lldpd08:24:032
161489943,0cyclictest3556-21lldpd08:19:032
161489943,0cyclictest25819-21awk07:52:532
161489943,0cyclictest2069-21awk08:47:532
161489943,0cyclictest20269-21awk08:12:542
161489943,0cyclictest13591-21awk08:07:542
161479943,0cyclictest3556-21lldpd10:34:031
161479943,0cyclictest3514-21snmpd10:47:591
161479943,0cyclictest3514-21snmpd10:19:441
161479943,0cyclictest3241-21irqbalance11:41:561
161479943,0cyclictest31546-21awk07:32:511
161479943,0cyclictest28383-21sadc12:07:371
161479943,0cyclictest23813-21awk09:22:531
161479943,0cyclictest23386-21awk09:42:531
161479943,0cyclictest23081-21awk10:02:531
161479943,0cyclictest15419-21awk08:57:501
161479943,0cyclictest15078-21awk09:57:521
161479943,0cyclictest12409-21sadc11:57:371
161469944,0cyclictest0-21swapper/010:10:230
161469943,0cyclictest7767-21awk08:27:530
161469943,0cyclictest30986-21kworker/0:010:17:430
161469943,0cyclictest24813-21awk07:27:530
161469943,0cyclictest1110-21awk08:22:530
169940,0watchdog/14433-21ssh09:48:321
169940,0watchdog/121778-21ssh09:20:331
169940,0watchdog/11756-21sh12:07:561
169940,0watchdog/113381-21ssh09:14:571
169940,0watchdog/112537-21ssh11:16:281
169940,0watchdog/110458-21ssh12:13:121
169940,0watchdog/10-21swapper/111:09:321
169940,0watchdog/10-21swapper/109:02:571
169940,0watchdog/10-21swapper/108:53:011
169940,0watchdog/10-21swapper/107:52:491
169940,0watchdog/10-21swapper/107:13:011
149940,0migration/06795-21sendmail_mailqu08:47:590
149940,0migration/04354-21runrttasks09:48:180
149940,0migration/024635-21kthreadcore09:02:560
149940,0migration/023421-21fschecks_time09:22:510
149940,0migration/021591-21kthreadcore08:12:550
149940,0migration/010481-21kthreadcore09:52:550
29725030,0irq/27-eth0-tx-29755-21aten2.4-expect07:07:500
29715030,0irq/26-eth0-rx-0-21swapper/308:55:353
29715030,0irq/26-eth0-rx-0-21swapper/308:47:493
29715030,0irq/26-eth0-rx-0-21swapper/308:47:243
29715030,0irq/26-eth0-rx-0-21swapper/308:02:363
29715030,0irq/26-eth0-rx-0-21swapper/307:28:013
29715030,0irq/26-eth0-rx-0-21swapper/307:14:133
27130,0ktimersoftd/20-21swapper/207:01:432
249930,0watchdog/20-21swapper/208:33:132
179930,0migration/17834-21kthreadcore08:02:571
179930,0migration/13557-21kthreadcore08:22:551
179930,0migration/129314-21aten_r1power_en11:27:481
161489932,0cyclictest5761-21awk07:37:532
161489932,0cyclictest3556-21lldpd07:24:312
161489932,0cyclictest26792-21tr07:27:562
161479933,0cyclictest3556-21lldpd07:39:331
161479932,0cyclictest7729-21irqstats09:32:561
161479932,0cyclictest3514-21snmpd07:43:581
161479932,0cyclictest23737-21gltestperf07:02:531
161479932,0cyclictest13489-21awk08:07:531
161479930,0cyclictest24342-21diskstats07:27:511
161469933,0cyclictest30986-21kworker/0:012:07:430
161469933,0cyclictest30986-21kworker/0:011:58:270
161469933,0cyclictest30986-21kworker/0:011:22:510
161469933,0cyclictest30986-21kworker/0:011:18:530
161469933,0cyclictest30986-21kworker/0:011:13:070
161469933,0cyclictest30986-21kworker/0:010:22:390
161469933,0cyclictest14047-21kworker/0:109:57:500
161469933,0cyclictest14047-21kworker/0:109:12:400
161469933,0cyclictest14047-21kworker/0:108:07:440
161469933,0cyclictest14047-21kworker/0:107:42:440
161469932,0cyclictest7411-21awk09:32:550
161469932,0cyclictest22713-21awk10:42:540
161469930,0cyclictest15046-21perl09:37:520
169930,0watchdog/10-21swapper/110:57:441
169930,0watchdog/10-21swapper/109:07:451
161479921,0cyclictest20189-21latency_hist08:37:391
169920,0watchdog/10-21swapper/108:47:491
169920,0watchdog/10-21swapper/108:42:411
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional