You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 07:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Mon Aug 17, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450640,0irq/25-eth10-21swapper/318:43:563
3149924332,0sleep00-21swapper/018:43:160
3149623425,0sleep20-21swapper/218:43:142
3161122723,0sleep10-21swapper/118:44:371
339960,0migration/38657-21sh23:12:343
339960,0migration/319275-21sh21:58:383
339960,0migration/317889-21sh22:59:493
339960,0migration/315982-21munin-node20:11:503
339960,0migration/31541-21munin-node22:27:023
318599960,0cyclictest30031-21diskmemload21:24:222
318589965,0cyclictest18651-21cut23:41:561
29715060,0irq/26-eth0-rx-8988-21date23:56:383
29715060,0irq/26-eth0-rx-8438-21nfs_client22:52:003
29715060,0irq/26-eth0-rx-14285-21iostat_ios21:36:553
149960,0migration/029375-21sh21:05:330
149960,0migration/021640-21fschecks_count21:41:520
149960,0migration/019168-21latency_hist22:41:380
149960,0migration/017755-21sh23:21:100
149960,0migration/017607-21fschecks_time00:01:520
149960,0migration/011788-21timerandwakeup21:12:020
8215050,0irq/24-0000:00:28046-21idleruntime-cro19:56:392
339950,0migration/39559-21needreboot22:11:573
339950,0migration/39197-21sendmail_mailqu22:32:023
339950,0migration/39054-21sh23:35:453
339950,0migration/37802-21taskset23:53:413
339950,0migration/36054-21kthreadcore20:01:573
339950,0migration/35700-21munin-node21:51:543
339950,0migration/35131-21kthreadcore19:36:593
339950,0migration/33030-21taskset20:23:543
339950,0migration/33006-21kthreadcore20:46:583
339950,0migration/328342-21munin-node19:31:563
339950,0migration/327989-21kthreadcore19:06:553
339950,0migration/327428-21taskset22:26:113
339950,0migration/326141-21aten_r1power_en23:26:513
339950,0migration/325771-21taskset23:04:313
339950,0migration/325295-21cpuspeed_turbos00:06:523
339950,0migration/325169-21munin-node19:02:003
339950,0migration/324170-21kthreadcore21:41:573
339950,0migration/322769-21kthreadcore23:41:573
339950,0migration/321918-21cpuspeed_turbos19:51:513
339950,0migration/32174-21taskset20:00:203
339950,0migration/317602-21munin-node20:36:543
339950,0migration/316686-21idleruntime-cro20:36:393
339950,0migration/316686-21idleruntime-cro20:36:393
339950,0migration/313917-21aten_r1power_po21:16:493
339950,0migration/313824-21latency_hist20:56:393
339950,0migration/313308-21munin-node19:17:033
318599955,0cyclictest0-21swapper/221:27:502
318599955,0cyclictest0-21swapper/200:02:112
318599950,0cyclictest0-21swapper/223:42:282
318589954,0cyclictest15833-21awk19:46:541
318579955,0cyclictest0-21swapper/000:07:350
318579950,0cyclictest0-21swapper/021:30:430
318579950,0cyclictest0-21swapper/021:09:540
29725050,0irq/27-eth0-tx-8346-21telnet19:41:500
29725050,0irq/27-eth0-tx-76492sleep200:14:522
29725050,0irq/27-eth0-tx-32068-21munin-node18:47:000
29725050,0irq/27-eth0-tx-29259-21ssh21:46:502
29725050,0irq/27-eth0-tx-26925-21idleruntime23:26:542
29725050,0irq/27-eth0-tx-23595-21munin-node20:42:000
29725050,0irq/27-eth0-tx-19933-21munin-node22:21:552
29725050,0irq/27-eth0-tx-0-21swapper/223:56:532
29725050,0irq/27-eth0-tx-0-21swapper/223:15:312
29725050,0irq/27-eth0-tx-0-21swapper/223:05:512
29725050,0irq/27-eth0-tx-0-21swapper/222:37:102
29725050,0irq/27-eth0-tx-0-21swapper/222:17:552
29725050,0irq/27-eth0-tx-0-21swapper/222:11:392
29725050,0irq/27-eth0-tx-0-21swapper/221:45:532
29725050,0irq/27-eth0-tx-0-21swapper/221:12:372
29725050,0irq/27-eth0-tx-0-21swapper/221:07:362
29725050,0irq/27-eth0-tx-0-21swapper/020:46:450
29725050,0irq/27-eth0-tx-0-21swapper/019:05:510
29725050,0irq/27-eth0-tx-0-21swapper/018:56:130
29715050,0irq/26-eth0-rx-31008-21munin-node21:06:563
29715050,0irq/26-eth0-rx-22596-21munin-node20:16:503
29715050,0irq/26-eth0-rx-1266-21sh00:11:533
29715050,0irq/26-eth0-rx-0-21swapper/323:50:383
29715050,0irq/26-eth0-rx-0-21swapper/323:40:183
29715050,0irq/26-eth0-rx-0-21swapper/322:38:323
29715050,0irq/26-eth0-rx-0-21swapper/321:50:023
29715050,0irq/26-eth0-rx-0-21swapper/321:29:313
29715050,0irq/26-eth0-rx-0-21swapper/321:21:543
29715050,0irq/26-eth0-rx-0-21swapper/321:15:233
29715050,0irq/26-eth0-rx-0-21swapper/300:03:413
259950,0migration/29463-21sh22:53:202
259950,0migration/25509-21kthreadcore20:01:572
259950,0migration/23430-21sh22:10:402
259950,0migration/231739-21sh00:09:002
259950,0migration/228193-21munin-node19:31:552
259950,0migration/22695-21timerwakeupswit20:22:022
259950,0migration/226843-21runrttasks22:02:362
259950,0migration/226809-21runrttasks22:46:042
259950,0migration/220037-21kthreadcore23:21:532
259950,0migration/215030-21irqrtprio21:16:542
259950,0migration/21361-21timerwakeupswit22:27:012
259950,0migration/211924-21kthreadcore23:16:542
249950,0watchdog/230268-21kthreadcore23:46:562
249950,0watchdog/20-21swapper/221:56:572
19150,0ktimersoftd/10-21swapper/123:02:541
179950,0migration/19198-21latency_hist20:06:391
179950,0migration/12454-21munin-node20:22:011
179950,0migration/12346-21gltestperf23:31:531
179950,0migration/118610-21kthreadcore21:16:551
179950,0migration/113386-21taskset19:17:081
179950,0migration/112052-21kthreadcore20:31:571
179950,0migration/112052-21kthreadcore20:31:571
149950,0migration/08902-21kthreadcore22:11:560
149950,0migration/07806-21taskset00:15:200
149950,0migration/07615-21kthreadcore20:51:560
149950,0migration/07577-21sh23:53:000
149950,0migration/05926-21kthreadcore22:51:580
149950,0migration/05733-21kthreadcore20:26:550
149950,0migration/04182-21kthreadcore20:01:560
149950,0migration/031671-21kthreadcore22:46:580
149950,0migration/030883-21kthreadcore23:26:570
149950,0migration/029753-21kthreadcore23:06:580
149950,0migration/026735-21kthreadcore21:21:560
149950,0migration/025415-21latency_hist23:46:380
149950,0migration/025092-21sh23:02:280
149950,0migration/02385-21latency_hist20:01:390
149950,0migration/023487-21kthreadcore19:26:560
149950,0migration/022131-21kthreadcore22:41:540
149950,0migration/021567-21kthreadcore23:41:570
149950,0migration/019195-21kthreadcore20:11:560
149950,0migration/012986-21cpuspeed_turbos21:56:510
149950,0migration/012460-21latency_hist21:56:380
149950,0migration/011310-21kthreadcore20:06:570
149950,0migration/010340-21sh22:34:220
339940,0migration/39645-21taskset20:27:273
339940,0migration/328236-21if_eth122:46:553
339940,0migration/321932-21kthreadcore23:21:543
339940,0migration/315721-21missed_timers23:16:563
339940,0migration/314848-21kthreadcore18:56:573
339940,0migration/312253-21timerwakeupswit18:52:013
339940,0migration/310684-21kthreadcore20:51:573
339940,0migration/310229-21kthreadcore19:41:563
318599943,0cyclictest6268-21awk21:31:532
318599943,0cyclictest3556-21lldpd19:21:422
318599943,0cyclictest3514-21snmpd20:41:492
318599943,0cyclictest3514-21snmpd20:40:492
318599943,0cyclictest3514-21snmpd20:16:492
318599943,0cyclictest3514-21snmpd19:56:492
318599943,0cyclictest1973-21awk23:51:532
318599943,0cyclictest11225-21awk20:31:562
318599943,0cyclictest11225-21awk20:31:562
318589943,0cyclictest7382-21awk18:51:531
318589943,0cyclictest455-21awk18:46:551
318589943,0cyclictest4392-21awk20:26:531
318589943,0cyclictest3556-21lldpd23:46:431
318589943,0cyclictest3556-21lldpd22:51:431
318589943,0cyclictest3556-21lldpd22:12:431
318589943,0cyclictest3556-21lldpd20:21:121
318589943,0cyclictest3556-21lldpd00:06:431
318589943,0cyclictest3556-21lldpd00:04:421
318589943,0cyclictest3514-21snmpd23:22:041
318589943,0cyclictest3514-21snmpd21:48:341
318589943,0cyclictest3514-21snmpd19:27:191
318589943,0cyclictest3241-21irqbalance23:36:561
318589943,0cyclictest3241-21irqbalance22:36:471
318589943,0cyclictest3241-21irqbalance22:29:371
318589943,0cyclictest3241-21irqbalance19:37:071
318589943,0cyclictest3241-21irqbalance00:12:561
318589943,0cyclictest27030-21awk23:26:551
318589943,0cyclictest17786-21awk20:36:551
318589943,0cyclictest14901-21awk19:21:551
318589942,0cyclictest3556-21lldpd23:16:431
318579944,0cyclictest0-21swapper/023:57:290
318579944,0cyclictest0-21swapper/020:21:110
318579943,0cyclictest6441-21tr23:31:570
318579943,0cyclictest28209-21awk19:31:550
318579943,0cyclictest21536-21awk22:01:560
318579943,0cyclictest1567-21awk19:11:530
318579943,0cyclictest14797-21awk21:16:530
318579943,0cyclictest14210-21awk21:36:540
318579943,0cyclictest13969-21awk18:56:560
318579940,0cyclictest0-21swapper/019:40:590
29725040,0irq/27-eth0-tx-4605-21cat21:51:382
29725040,0irq/27-eth0-tx-3524-21sshd19:50:070
29725040,0irq/27-eth0-tx-1568-21ssh23:31:422
29725040,0irq/27-eth0-tx-0-21swapper/223:36:442
29725040,0irq/27-eth0-tx-0-21swapper/222:56:482
29725040,0irq/27-eth0-tx-0-21swapper/222:46:512
29725040,0irq/27-eth0-tx-0-21swapper/222:31:502
29725040,0irq/27-eth0-tx-0-21swapper/221:01:522
29725040,0irq/27-eth0-tx-0-21swapper/220:56:412
29725040,0irq/27-eth0-tx-0-21swapper/020:36:070
29725040,0irq/27-eth0-tx-0-21swapper/020:36:070
29725040,0irq/27-eth0-tx-0-21swapper/019:52:010
29725040,0irq/27-eth0-tx-0-21swapper/019:07:020
29715040,0irq/26-eth0-rx-0-21swapper/323:09:153
29715040,0irq/26-eth0-rx-0-21swapper/322:41:553
29715040,0irq/26-eth0-rx-0-21swapper/322:17:343
29715040,0irq/26-eth0-rx-0-21swapper/322:06:433
29715040,0irq/26-eth0-rx-0-21swapper/322:01:573
29715040,0irq/26-eth0-rx-0-21swapper/321:31:493
29715040,0irq/26-eth0-rx-0-21swapper/321:01:423
29715040,0irq/26-eth0-rx-0-21swapper/320:06:573
259940,0migration/29794-21fschecks_time20:06:532
259940,0migration/29302-21kthreadcore20:51:562
259940,0migration/27780-21munin-node19:16:532
259940,0migration/27690-21runrttasks19:37:192
259940,0migration/23694-21kthreadcore19:11:552
259940,0migration/230863-21kthreadcore23:06:582
259940,0migration/228478-21kthreadcore19:06:562
249940,0watchdog/232416-21kthreadcore20:46:572
249940,0watchdog/219805-21kthreadcore19:46:572
249940,0watchdog/20-21swapper/221:36:532
249940,0watchdog/20-21swapper/220:14:052
249940,0watchdog/20-21swapper/219:42:012
249940,0watchdog/20-21swapper/219:26:532
249940,0watchdog/20-21swapper/219:01:572
249940,0watchdog/20-21swapper/218:57:012
19140,0ktimersoftd/10-21swapper/121:04:581
179940,0migration/16869-21irqcore21:11:551
179940,0migration/129017-21runrttasks21:25:321
179940,0migration/127449-21irqcore19:06:551
179940,0migration/127165-21latency_hist22:46:381
179940,0migration/123952-21kthreadcore19:51:561
179940,0migration/120266-21munin-node22:41:521
179940,0migration/116379-21fschecks_count20:11:521
179940,0migration/114785-21kthreadcore18:56:571
179940,0migration/113979-21kernelversion21:56:561
179940,0migration/113466-21kthreadcore23:56:571
179940,0migration/113024-21munin-node22:16:531
169940,0watchdog/17217-21tr20:01:571
169940,0watchdog/14617-21kthreadcore23:51:561
169940,0watchdog/14606-21irqcore22:31:561
169940,0watchdog/13983-21ssh21:07:171
169940,0watchdog/132414-21kthreadcore20:46:571
169940,0watchdog/128195-21grep20:41:571
169940,0watchdog/12637-21diskstats23:11:521
169940,0watchdog/118188-21cut21:36:571
169940,0watchdog/117265-21ssh22:57:561
169940,0watchdog/10-21swapper/122:21:571
169940,0watchdog/10-21swapper/121:42:011
169940,0watchdog/10-21swapper/121:32:011
169940,0watchdog/10-21swapper/121:29:371
169940,0watchdog/10-21swapper/119:31:571
149940,0migration/09942-21cpuspeed_turbos23:36:510
149940,0migration/028814-21df_abs22:06:520
149940,0migration/010582-21kthreadcore21:31:550
339930,0migration/321100-21runrttasks19:48:173
318599932,0cyclictest7295-21awk18:51:532
318599932,0cyclictest3514-21snmpd20:26:492
318589933,0cyclictest0-21swapper/120:51:441
318589932,0cyclictest3556-21lldpd19:41:431
318589932,0cyclictest3241-21irqbalance19:11:571
318589932,0cyclictest30031-21diskmemload22:01:541
318589932,0cyclictest29354-21awk22:06:551
318589932,0cyclictest29185-21awk19:56:551
318589932,0cyclictest20568-21awk19:01:551
318589932,0cyclictest14342-21grep20:56:521
318579933,0cyclictest7679-21kworker/0:021:46:480
318579933,0cyclictest7679-21kworker/0:020:36:420
318579933,0cyclictest7679-21kworker/0:020:21:400
318579933,0cyclictest7679-21kworker/0:019:21:500
318579933,0cyclictest7679-21kworker/0:019:16:400
318579933,0cyclictest6783-21kworker/0:023:12:060
318579933,0cyclictest6783-21kworker/0:022:56:480
318579933,0cyclictest29242-21kworker/0:222:21:480
318579932,0cyclictest28770-21awk22:26:530
318579932,0cyclictest19186-21awk23:21:520
318579932,0cyclictest14796-21awk20:56:530
318579932,0cyclictest12877-21awk22:16:520
29715030,0irq/26-eth0-rx-0-21swapper/320:41:513
29715030,0irq/26-eth0-rx-0-21swapper/319:26:543
29715030,0irq/26-eth0-rx-0-21swapper/319:21:513
29715030,0irq/26-eth0-rx-0-21swapper/318:50:483
249930,0watchdog/20-21swapper/218:46:572
169930,0watchdog/10-21swapper/123:06:401
169930,0watchdog/10-21swapper/121:52:371
29715020,0irq/26-eth0-rx-0-21swapper/319:11:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional