You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 02:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon May 04, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450460,0irq/25-eth10-21swapper/318:53:163
580323425,0sleep10-21swapper/118:55:511
569523425,0sleep20-21swapper/218:54:362
549523430,0sleep00-21swapper/018:52:130
339960,0migration/319151-21sh21:42:073
339960,0migration/318723-21sh23:24:413
339960,0migration/31831-21sh23:33:183
339960,0migration/317182-21sh00:22:283
29715060,0irq/26-eth0-rx-23338-21if_eth020:21:553
149960,0migration/07137-21kthreadcore20:56:560
149960,0migration/04174-21latency_hist20:56:390
149960,0migration/031837-21sh19:12:010
149960,0migration/027864-21sh22:30:460
149960,0migration/02747-21sh23:56:330
149960,0migration/019008-21sh23:25:300
149960,0migration/016354-21fschecks_time20:16:520
149960,0migration/011999-21sh22:21:080
8215050,0irq/24-0000:00:20641-21cat22:06:382
59459955,0cyclictest0-21swapper/223:24:352
59459955,0cyclictest0-21swapper/221:15:082
59459950,0cyclictest0-21swapper/223:07:232
59459950,0cyclictest0-21swapper/221:29:202
59459950,0cyclictest0-21swapper/221:22:142
59439950,0cyclictest0-21swapper/123:24:261
59429950,0cyclictest0-21swapper/022:21:410
339950,0migration/39262-21munin-node19:46:553
339950,0migration/332513-21kthreadcore21:31:563
339950,0migration/331801-21kthreadcore22:11:553
339950,0migration/331721-21proc_pri19:11:593
339950,0migration/331221-21kthreadcore21:51:563
339950,0migration/328183-21munin-node23:51:523
339950,0migration/325583-21taskset19:07:373
339950,0migration/322578-21kthreadcore19:31:553
339950,0migration/321245-21munin-node21:06:503
339950,0migration/32084-21taskset00:15:323
339950,0migration/320128-21taskset19:30:273
339950,0migration/319274-21kthreadcore20:16:563
339950,0migration/317590-21kthreadcore22:21:583
339950,0migration/31702-21df_abs19:41:513
339950,0migration/316599-21kthreadcore19:51:533
339950,0migration/316462-21kthreadcore22:41:563
339950,0migration/314427-21kthreadcore20:36:563
339950,0migration/31367-21latency22:31:573
339950,0migration/31207-21munin-node19:16:533
339950,0migration/310732-21sh22:17:253
339950,0migration/310624-21timerandwakeup21:57:013
29725050,0irq/27-eth0-tx-9259-21timerandwakeup23:37:012
29725050,0irq/27-eth0-tx-6163-21munin-node18:56:540
29725050,0irq/27-eth0-tx-4117-21munin-node22:36:572
29725050,0irq/27-eth0-tx-28897-21aten_r1power_po22:11:492
29725050,0irq/27-eth0-tx-27069-21ssh23:51:162
29725050,0irq/27-eth0-tx-20824-21munin-node21:06:492
29725050,0irq/27-eth0-tx-19560-21sed23:26:382
29725050,0irq/27-eth0-tx-13392-21irqrtprio22:41:552
29725050,0irq/27-eth0-tx-12461-21munin-node22:22:002
29725050,0irq/27-eth0-tx-0-21swapper/222:48:232
29725050,0irq/27-eth0-tx-0-21swapper/222:18:182
29725050,0irq/27-eth0-tx-0-21swapper/200:02:242
29725050,0irq/27-eth0-tx-0-21swapper/020:01:580
29725050,0irq/27-eth0-tx-0-21swapper/019:43:030
29715050,0irq/26-eth0-rx-9710-21ntp_offset22:36:583
29715050,0irq/26-eth0-rx-4891-21ssh21:16:413
29715050,0irq/26-eth0-rx-4502-21fschecks_time22:56:513
29715050,0irq/26-eth0-rx-21272-21kthreadcore00:06:553
29715050,0irq/26-eth0-rx-1842-21unixbench_multi22:52:023
29715050,0irq/26-eth0-rx-18035-21vmstat23:02:033
29715050,0irq/26-eth0-rx-17779-21ssh00:03:283
29715050,0irq/26-eth0-rx-14211-21iostat21:21:573
29715050,0irq/26-eth0-rx-0-21swapper/323:56:563
29715050,0irq/26-eth0-rx-0-21swapper/323:41:563
29715050,0irq/26-eth0-rx-0-21swapper/323:16:553
29715050,0irq/26-eth0-rx-0-21swapper/322:27:223
29715050,0irq/26-eth0-rx-0-21swapper/322:08:073
29715050,0irq/26-eth0-rx-0-21swapper/322:04:543
29715050,0irq/26-eth0-rx-0-21swapper/321:46:573
29715050,0irq/26-eth0-rx-0-21swapper/321:37:113
29715050,0irq/26-eth0-rx-0-21swapper/321:30:583
29715050,0irq/26-eth0-rx-0-21swapper/321:01:513
29715050,0irq/26-eth0-rx-0-21swapper/300:16:553
259950,0migration/29309-21sh23:57:032
259950,0migration/27293-21kthreadcore23:16:552
259950,0migration/226124-21latency_hist19:11:402
259950,0migration/215789-21kthreadcore21:41:552
259950,0migration/214790-21kthreadcore22:01:552
259950,0migration/211785-21sh21:59:432
259950,0migration/211362-21sh23:00:472
249950,0watchdog/22684-21cat21:32:012
249950,0watchdog/20-21swapper/220:16:532
179950,0migration/19975-21sh00:19:511
179950,0migration/19607-21taskset23:57:571
179950,0migration/16171-21aten_r1power_cu18:56:491
179950,0migration/15946-21kthreadcore22:36:551
179950,0migration/123903-21kthreadcore19:56:571
179950,0migration/112235-21latency_hist22:21:381
149950,0migration/09808-21sh23:37:530
149950,0migration/09651-21kthreadcore21:16:580
149950,0migration/08431-21kthreadcore21:36:560
149950,0migration/07669-21kthreadcore22:36:550
149950,0migration/04122-21sh21:35:080
149950,0migration/030943-21kthreadcore19:36:560
149950,0migration/030247-21kthreadcore22:11:540
149950,0migration/028220-21sh22:51:500
149950,0migration/027041-21switchtime21:27:010
149950,0migration/026575-21munin-node22:06:590
149950,0migration/026375-21runrttasks00:10:070
149950,0migration/026078-21kthreadcore20:46:550
149950,0migration/023742-21kthreadcore19:56:560
149950,0migration/02212-21kthreadcore19:16:560
149950,0migration/021659-21kthreadcore20:41:560
149950,0migration/021045-21munin-node22:46:530
149950,0migration/019922-21taskset19:28:010
149950,0migration/019077-21unixbench_singl21:42:020
149950,0migration/018088-21timerandwakeup22:42:010
149950,0migration/017231-21kthreadcore21:21:580
149950,0migration/016073-21kthreadcore00:01:550
149950,0migration/012555-21sh21:01:440
149950,0migration/012480-21kthreadcore20:11:550
149950,0migration/010516-21kthreadcore19:46:560
59459944,0cyclictest0-21swapper/219:46:042
59459943,0cyclictest8097-21awk19:21:552
59459943,0cyclictest3556-21lldpd20:43:312
59459943,0cyclictest3556-21lldpd20:32:312
59459943,0cyclictest3556-21lldpd20:29:012
59459943,0cyclictest3556-21lldpd20:08:012
59459943,0cyclictest3556-21lldpd19:47:302
59459943,0cyclictest3556-21lldpd19:32:302
59459943,0cyclictest3556-21lldpd18:57:302
59439943,0cyclictest4606-21awk23:16:531
59439943,0cyclictest4361-21awk20:31:551
59439943,0cyclictest3514-21snmpd23:41:471
59439943,0cyclictest3514-21snmpd22:46:311
59439943,0cyclictest3514-21snmpd21:24:011
59439943,0cyclictest3514-21snmpd20:48:311
59439943,0cyclictest3241-21irqbalance00:03:371
59439943,0cyclictest30119-21ps22:11:541
59439943,0cyclictest29995-21awk21:11:531
59439943,0cyclictest28033-21awk19:36:541
59439943,0cyclictest27021-21awk19:11:541
59439943,0cyclictest25632-21grep23:06:581
59439943,0cyclictest21717-21awk21:06:521
59439943,0cyclictest20477-21sadc22:06:381
59439943,0cyclictest11126-21awk20:36:541
59429944,0cyclictest0-21swapper/021:46:410
59429943,0cyclictest5626-21awk21:56:530
59429943,0cyclictest4099-21awk23:56:550
59429943,0cyclictest3556-21lldpd19:54:310
59429943,0cyclictest3556-21lldpd19:26:300
59429943,0cyclictest3514-21snmpd19:02:160
59429943,0cyclictest3514-21snmpd00:23:470
59429943,0cyclictest20368-21awk19:06:530
59429943,0cyclictest19161-21sadc23:46:380
59429940,0cyclictest2367-21munin-node20:06:520
339940,0migration/38735-21kthreadcore19:21:573
339940,0migration/38653-21kthreadcore20:56:563
339940,0migration/38463-21kthreadcore20:31:573
339940,0migration/36464-21kthreadcore20:06:573
339940,0migration/331188-21kthreadcore19:36:563
339940,0migration/331163-21kthreadcore20:26:553
339940,0migration/329450-21kthreadcore23:11:553
339940,0migration/31386-21kthreadcore20:51:593
339940,0migration/310414-21irqstats20:11:543
29725040,0irq/27-eth0-tx-9022-21ssh00:17:052
29725040,0irq/27-eth0-tx-22449-21munin-node20:21:580
29725040,0irq/27-eth0-tx-0-21swapper/223:52:012
29725040,0irq/27-eth0-tx-0-21swapper/222:51:482
29725040,0irq/27-eth0-tx-0-21swapper/222:32:392
29725040,0irq/27-eth0-tx-0-21swapper/222:26:532
29725040,0irq/27-eth0-tx-0-21swapper/221:47:172
29725040,0irq/27-eth0-tx-0-21swapper/221:37:172
29725040,0irq/27-eth0-tx-0-21swapper/221:01:432
29725040,0irq/27-eth0-tx-0-21swapper/220:58:322
29725040,0irq/27-eth0-tx-0-21swapper/200:07:022
29725040,0irq/27-eth0-tx-0-21swapper/020:36:500
29715040,0irq/26-eth0-rx-0-21swapper/323:46:403
29715040,0irq/26-eth0-rx-0-21swapper/323:36:413
29715040,0irq/26-eth0-rx-0-21swapper/323:27:483
29715040,0irq/26-eth0-rx-0-21swapper/323:07:283
29715040,0irq/26-eth0-rx-0-21swapper/322:48:273
29715040,0irq/26-eth0-rx-0-21swapper/321:13:313
29715040,0irq/26-eth0-rx-0-21swapper/318:56:543
259940,0migration/231255-21kthreadcore00:11:552
259940,0migration/229145-21kthreadcore23:31:552
259940,0migration/227944-21latency_hist23:11:382
259940,0migration/223728-21kernelversion20:21:562
259940,0migration/221098-21kthreadcore19:06:552
259940,0migration/217074-21taskset00:22:092
259940,0migration/215500-21kthreadcore23:01:562
259940,0migration/214580-21kthreadcore23:41:562
259940,0migration/213771-21latency_hist19:26:402
259940,0migration/21164-21munin-node20:01:592
249940,0watchdog/228998-21ssh21:51:452
249940,0watchdog/218634-21sed19:02:022
249940,0watchdog/215452-21missed_timers20:36:572
249940,0watchdog/20-21swapper/221:16:572
249940,0watchdog/20-21swapper/219:51:532
179940,0migration/18673-21kthreadcore21:36:561
179940,0migration/17163-21runrttasks19:21:501
179940,0migration/15059-21fschecks_time20:56:521
179940,0migration/130597-21kthreadcore21:51:561
179940,0migration/130543-21kthreadcore23:31:551
179940,0migration/126271-21switchtime22:27:031
179940,0migration/122924-21fschecks_count20:21:531
179940,0migration/121941-21kthreadcore19:06:551
179940,0migration/121599-21kthreadcore23:46:551
179940,0migration/121300-21kthreadcore23:26:561
179940,0migration/119084-21runrttasks19:04:251
179940,0migration/117256-21fschecks_count20:41:511
179940,0migration/116545-21kthreadcore19:51:531
179940,0migration/11598-21kthreadcore20:51:591
179940,0migration/112679-21aten_r1power_fr21:01:491
179940,0migration/112271-21kthreadcore19:46:571
169940,0watchdog/19926-21ssh22:57:011
169940,0watchdog/19163-21smartctl23:37:011
169940,0watchdog/16559-21runrttasks22:32:011
169940,0watchdog/132420-21grep23:51:571
169940,0watchdog/126768-21ssh00:11:161
169940,0watchdog/12272-21missed_timers21:31:571
169940,0watchdog/121953-21irqrtprio22:06:531
169940,0watchdog/118433-21missed_timers21:41:571
169940,0watchdog/10-21swapper/123:01:571
169940,0watchdog/10-21swapper/121:16:571
169940,0watchdog/10-21swapper/120:26:491
149940,0migration/04407-21sh21:16:020
149940,0migration/030347-21kernelversion21:51:550
149940,0migration/023569-21kthreadcore23:46:560
149940,0migration/02187-21sh23:34:210
149940,0migration/014477-21kthreadcore22:01:550
59459933,0cyclictest3556-21lldpd20:47:312
59459933,0cyclictest3556-21lldpd19:18:002
59459932,0cyclictest3556-21lldpd20:12:012
59459930,0cyclictest0-21swapper/219:36:592
59439933,0cyclictest10049-21awk20:11:531
59439932,0cyclictest3514-21snmpd00:12:171
59439932,0cyclictest29224-21awk22:51:541
59439932,0cyclictest21448-21awk19:31:541
59439932,0cyclictest20223-21aten_r1power_en22:46:491
59439932,0cyclictest1246-21awk19:16:541
59439931,0cyclictest5251-21fschecks_time21:56:521
59439930,0cyclictest0-21swapper/120:07:311
59439930,0cyclictest0-21swapper/119:43:051
59429933,0cyclictest24453-21kworker/0:221:06:400
59429933,0cyclictest21980-21kworker/0:223:27:060
59429933,0cyclictest21980-21kworker/0:223:16:500
59429933,0cyclictest21980-21kworker/0:223:01:480
59429933,0cyclictest21980-21kworker/0:222:56:420
59429933,0cyclictest21980-21kworker/0:222:32:480
59429933,0cyclictest21980-21kworker/0:200:16:440
59429933,0cyclictest21980-21kworker/0:200:11:540
59429932,0cyclictest21058-21sed23:06:540
59429930,0cyclictest28675-21perl23:11:510
59429930,0cyclictest0-21swapper/019:31:520
29725030,0irq/27-eth0-tx-0-21swapper/020:31:530
29725030,0irq/27-eth0-tx-0-21swapper/020:26:500
29715030,0irq/26-eth0-rx-0-21swapper/320:46:493
29715030,0irq/26-eth0-rx-0-21swapper/320:46:003
29715030,0irq/26-eth0-rx-0-21swapper/320:02:023
29715030,0irq/26-eth0-rx-0-21swapper/319:01:503
259930,0migration/222107-21fschecks_time19:56:532
249930,0watchdog/20-21swapper/220:51:572
179930,0migration/127901-21runrttasks21:28:361
169930,0watchdog/14156-21diskmemload00:21:561
169930,0watchdog/128125-21munin-node20:01:531
169930,0watchdog/10-21swapper/123:13:171
169930,0watchdog/10-21swapper/122:21:571
169930,0watchdog/10-21swapper/121:46:491
169930,0watchdog/10-21swapper/120:16:531
169930,0watchdog/10-21swapper/119:26:581
35120,0ktimersoftd/327553-21runrttasks19:57:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional