You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 07:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun May 10, 2026 00:43:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2298431995341,9cyclictest2320936-21systemd-run20:25:011
2298430995227,22cyclictest2299835-21head19:11:050
2298431995037,10cyclictest2387785-21sshd22:15:521
2298431995037,10cyclictest2387785-21sshd22:15:521
2298430995035,12cyclictest2454038-21cron23:40:010
2298430994933,14cyclictest2470025-21awk00:00:010
2298432994737,4cyclictest0-21swapper/220:15:542
2298431994737,6cyclictest2391321-21fschecks_count22:20:431
229843099472,18cyclictest2399721-21ntpq22:30:500
2298432994637,5cyclictest0-21swapper/223:30:522
229843299463,39cyclictest2343813-21ssh21:20:502
2298431994638,3cyclictest0-21swapper/100:00:011
2298430994628,15cyclictest439-21systemd-logind19:35:010
2298432994538,3cyclictest0-21swapper/223:05:502
2298432994535,4cyclictest0-21swapper/221:15:532
2298430994527,15cyclictest2359108-21/usr/sbin/munin21:40:460
229843099450,12cyclictest2323318-21ntpq20:30:510
2298432994436,5cyclictest2438197-21latency_hist23:20:022
2298432994435,6cyclictest2302947-21systemd-run19:25:012
2298432994413,10cyclictest2425678-21ssh23:04:002
2298431994437,3cyclictest0-21swapper/100:05:421
2298431994435,6cyclictest2471880-21unixbench-2d00:01:041
2298431994435,6cyclictest2471880-21unixbench-2d00:01:041
229843099446,35cyclictest2341660-21ssh21:18:240
2298430994432,9cyclictest2439466-21sshd23:20:520
2298430994430,11cyclictest2301464-21latency_hist19:20:010
2298430994424,17cyclictest2380434-21unixbench_singl22:06:020
2298432994333,7cyclictest2455373-21ntpq23:40:512
229843299433,34cyclictest2387885-21ssh22:15:562
229843299433,34cyclictest2387885-21ssh22:15:562
2298432994331,8cyclictest420-21avahi-daemon21:30:462
229843299432,31cyclictest2375941-21ssh22:00:572
2298432994313,13cyclictest421-21dbus-daemon21:08:342
2298431994335,6cyclictest1054-21mta-sts-daemon23:51:431
2298431994323,16cyclictest271ktimers/121:25:471
229843099438,3cyclictest2461130-21ssh23:48:160
229843099436,3cyclictest2368549-21ssh21:51:120
229843099432,7cyclictest2411596-21ntpq22:45:510
229843099432,36cyclictest2343864-21ntpq21:20:520
2298430994313,26cyclictest0-21swapper/020:56:000
2298430994313,26cyclictest0-21swapper/020:56:000
2298432994232,7cyclictest2366514-21munin-plugin-st21:50:012
2298432994232,7cyclictest2366514-21munin-plugin-st21:50:012
2298431994234,5cyclictest2307455-21latency_hist19:40:021
229843099423,6cyclictest2318853-21ntpq20:15:540
2298430994233,5cyclictest0-21swapper/023:20:010
229843099423,15cyclictest2356050-21ssh21:36:010
229843099422,36cyclictest2419612-21ssh22:55:570
2298430994218,21cyclictest0-21swapper/023:10:510
229843099421,12cyclictest2417818-21ssh22:54:080
229843099421,11cyclictest2332245-21ntpq21:00:490
229843099420,13cyclictest2349599-21ssh21:28:030
2298432994112,10cyclictest2431431-21ntpq23:10:482
2298430994128,11cyclictest2422885-21aten_r2power_po23:00:400
2298430994127,7cyclictest12350irq/32-eth022:15:500
2298430994127,7cyclictest12350irq/32-eth022:15:500
229843099412,6cyclictest2326393-21ntpq20:40:550
229843099412,6cyclictest2314300-21ntpq20:00:490
229843099412,15cyclictest2479308-21awk00:10:510
2298430994119,19cyclictest2471958-21head00:01:040
2298430994119,19cyclictest2471958-21head00:01:040
2298430994119,17cyclictest0-21swapper/000:35:030
2298430994116,19cyclictest496353-21cups-browsed22:40:520
229843099411,12cyclictest2329239-21ntpq20:50:520
229843299409,12cyclictest2294555-21kworker/u8:2+xprtiod19:40:032
2298432994030,6cyclictest2396312-21unixbench_multi22:26:012
2298432994013,7cyclictest2350351-21ssh21:29:302
2298431994033,4cyclictest2319413-21basename20:20:011
2298431994033,3cyclictest0-21swapper/121:15:011
2298431994032,5cyclictest2494771-21ssh00:30:431
2298431994031,6cyclictest2313380-21sshd19:59:521
229843099407,4cyclictest2316887-21df20:10:410
229843099404,7cyclictest2311330-21ntpq19:50:510
2298430994033,4cyclictest1056-21mta-sts-daemon21:10:530
229843099402,8cyclictest2483354-21ssh00:15:560
229843099402,6cyclictest2309758-21ntpq19:45:480
229843099402,6cyclictest2308335-21ntpq19:40:510
229843099402,5cyclictest2427554-21ntpq23:05:530
229843099402,4cyclictest2324755-21ntpq20:35:480
229843099401,9cyclictest2463298-21ntpq23:50:490
229843099401,9cyclictest2320297-21ntpq20:20:490
229843099401,8cyclictest2456389-21ssh23:41:480
229843099401,6cyclictest2306748-21ntpq19:35:490
229843099401,4cyclictest2365074-21ssh21:47:120
229843099401,4cyclictest2365074-21ssh21:47:120
2298430994013,24cyclictest0-21swapper/023:25:440
229843099401,11cyclictest2398211-21sh22:29:370
229843099401,10cyclictest2486223-21ssh00:20:080
229843099401,10cyclictest2405216-21ssh22:37:450
2298432993932,4cyclictest0-21swapper/223:45:512
2298432993932,3cyclictest0-21swapper/222:20:512
2298432993931,5cyclictest2374604-21latency_hist22:00:022
2298432993931,5cyclictest2368366-21unixbench_multi21:51:002
2298432993930,4cyclictest0-21swapper/222:30:442
2298432993915,7cyclictest2334402-21diskmemload00:20:192
2298432993913,8cyclictest2312840-21ntpq19:55:532
2298432993911,8cyclictest2439569-21ntpq23:20:572
2298432993910,10cyclictest2322407-21munin-run20:30:012
2298431993929,6cyclictest2436001-21unixbench-2d23:16:021
229843099399,2cyclictest0-21swapper/019:25:520
229843099399,2cyclictest0-21swapper/019:25:520
229843099392,6cyclictest2391863-21ntpq22:20:540
229843099392,5cyclictest2475330-21ntpq00:05:520
229843099392,5cyclictest2333823-21ntpq21:05:540
229843099392,19cyclictest2499118-21ssh00:35:490
229843099391,7cyclictest2302270-21ntpq19:20:490
229843099391,6cyclictest2312777-21ntpq19:55:500
229843099391,11cyclictest2447428-21ntpq23:30:530
229843099391,11cyclictest2327774-21ntpq20:45:510
229843099390,5cyclictest2377536-21ssh22:03:100
2298432993831,4cyclictest0-21swapper/221:10:522
2298432993829,6cyclictest2499416-21mailstats00:35:572
2298432993813,9cyclictest2323898-21munin-run20:35:012
2298432993813,7cyclictest2303839-21awk19:25:512
2298432993813,7cyclictest2303839-21awk19:25:512
2298432993812,11cyclictest2410631-21sh22:45:122
2298432993811,9cyclictest2480260-21ssh00:11:402
2298431993830,3cyclictest0-21swapper/123:35:011
2298431993827,8cyclictest2399512-21latency22:30:451
229843099382,32cyclictest421-21dbus-daemon21:59:360
229843099381,8cyclictest2321800-21ntpq20:25:500
2298430993815,20cyclictest2384383-21unixbench_singl22:11:030
2298430993814,21cyclictest0-21swapper/020:05:490
229843099381,12cyclictest2351643-21ssh21:30:490
229843299371,31cyclictest2463300-21ssh23:50:492
2298432993712,8cyclictest2408790-21ssh22:41:542
2298432993712,7cyclictest2334402-21diskmemload23:38:372
2298432993711,9cyclictest2474177-21date00:05:022
2298432993711,9cyclictest2474177-21date00:05:022
2298431993730,4cyclictest0-21swapper/121:40:011
2298431993729,5cyclictest2426186-21sh23:05:001
2298431993729,4cyclictest0-21swapper/122:55:571
229843199371,5cyclictest171rcu_preempt19:50:521
229843099371,9cyclictest2491339-21ntpq00:25:560
229843299369,8cyclictest2421365-21ssh22:58:222
229843299367,12cyclictest2325774-21aten2.4-expect20:40:402
2298432993623,10cyclictest2418013-21ssh22:54:312
2298432993611,8cyclictest2490158-21ssh00:25:042
2298432993611,8cyclictest2332377-21ntpq21:00:562
2298432993611,8cyclictest2299297-21ntpq19:10:542
2298432993610,9cyclictest2314097-21sed20:00:442
2298431993629,3cyclictest0-21swapper/122:45:021
229843199362,24cyclictest2367901-21ssh21:50:531
2298432993528,4cyclictest0-21swapper/200:20:012
2298432993528,3cyclictest0-21swapper/223:30:022
2298432993527,4cyclictest0-21swapper/223:56:002
2298432993511,7cyclictest2315402-21df_inode20:05:402
2298431993527,5cyclictest2359135-21aten_r2power_cu21:40:421
2298431993526,6cyclictest420-21avahi-daemon19:20:581
2298431993518,12cyclictest1056-21mta-sts-daemon22:35:551
229843299349,9cyclictest2329782-21awk20:51:062
229843299349,9cyclictest2309775-21ntpq19:45:482
2298432993425,5cyclictest2406369-21sendmail22:40:012
2298432993424,6cyclictest496353-21cups-browsed20:49:272
2298432993423,7cyclictest420-21avahi-daemon00:05:092
2298432993418,11cyclictest2385184-21ssh22:12:322
229843199343,23cyclictest2415558-21ssh22:50:481
2298431993417,12cyclictest421-21dbus-daemon00:20:551
229843299339,4cyclictest2324784-21ntpq20:35:502
2298432993324,6cyclictest0-21swapper/219:10:012
2298432993322,7cyclictest2308394-21ntpq19:40:542
2298432993318,11cyclictest2494295-21ssh00:30:222
2298432993313,6cyclictest2357331-21diskmemload21:37:392
2298432993310,7cyclictest2305576-21cat19:31:012
2298431993326,4cyclictest0-21swapper/123:30:011
2298431993326,4cyclictest0-21swapper/122:05:011
2298431993325,5cyclictest2332778-21unixbench_multi21:01:021
2298431993325,3cyclictest0-21swapper/121:45:571
2298431993325,3cyclictest0-21swapper/121:45:571
2298431993324,6cyclictest2344458-21unixbench_singl21:21:021
2298431993322,7cyclictest496353-21cups-browsed22:08:351
2298431993322,7cyclictest496353-21cups-browsed21:30:461
229843199332,26cyclictest261-21systemd-journal00:10:521
2298432993220,8cyclictest2359976-21ntpq21:40:572
2298432993210,7cyclictest2381243-21diskmemload22:07:382
229843199328,17cyclictest420-21avahi-daemon20:40:511
2298431993224,5cyclictest0-21swapper/123:40:011
2298431993224,4cyclictest0-21swapper/121:15:451
2298431993223,6cyclictest2328414-21systemd-run20:50:011
2298431993223,5cyclictest2396297-21grep22:26:011
229843199322,23cyclictest2383948-21ssh22:10:581
229843299316,8cyclictest2320343-21ntpq20:20:512
229843299314,21cyclictest2311364-21sshd19:50:532
2298432993110,4cyclictest2330780-21ntpq20:55:562
2298432993110,4cyclictest2330780-21ntpq20:55:562
2298431993124,4cyclictest2485956-21sh00:20:011
229843199312,23cyclictest2459311-21ssh23:45:501
2298431993121,7cyclictest2333641-21missed_timers21:05:491
2298432993022,5cyclictest2301308-21unixbench_multi19:16:062
2298432993015,10cyclictest2317281-21ntpq20:10:522
2298431993023,4cyclictest2430212-21systemd-run23:10:011
2298431993022,5cyclictest2499095-21ntp_kernel_err00:35:481
2298431993022,3cyclictest0-21swapper/121:55:501
2298431992922,4cyclictest2304334-21unixbench_singl19:26:011
2298431992922,4cyclictest2304334-21unixbench_singl19:26:011
2298431992921,5cyclictest2442125-21systemd-run23:25:011
2298431992921,5cyclictest2300421-21fschecks_count19:15:471
2298431992921,4cyclictest0-21swapper/120:30:011
2298430992912,14cyclictest0-21swapper/019:10:010
2298431992822,3cyclictest0-21swapper/122:45:491
229843199282,21cyclictest2491350-21ssh00:25:571
2298431992821,4cyclictest2329894-21/usr/sbin/munin20:55:011
2298431992821,4cyclictest2325438-21systemd20:40:021
2298431992817,7cyclictest620-21sshd19:10:531
229843199281,22cyclictest2308377-21sshd19:40:531
2298431992721,3cyclictest0-21swapper/121:00:001
2298431992721,3cyclictest0-21swapper/121:00:001
2298431992719,5cyclictest2317785-21unixbench_multi20:11:041
2298431992718,7cyclictest2454788-21cpu23:40:401
229843199270,5cyclictest171rcu_preempt23:10:501
229843199260,2cyclictest171rcu_preempt20:00:481
2298431992520,2cyclictest0-21swapper/120:35:011
2298431992518,4cyclictest1055-21mta-sts-daemon19:31:031
2298431992518,4cyclictest0-21swapper/119:50:011
229843199251,18cyclictest496353-21cups-browsed20:05:481
2298431992014,3cyclictest0-21swapper/119:10:011
229843399183,7cyclictest0-21swapper/322:20:523
229843399172,8cyclictest0-21swapper/323:50:493
229843399172,8cyclictest0-21swapper/321:35:563
229843399172,7cyclictest0-21swapper/300:30:553
229843399161,9cyclictest0-21swapper/323:20:513
229843399153,9cyclictest0-21swapper/323:25:483
229843399153,7cyclictest0-21swapper/300:10:523
229843399153,6cyclictest0-21swapper/322:10:513
229843399152,7cyclictest0-21swapper/320:25:543
229843399151,8cyclictest0-21swapper/323:55:533
229843399151,8cyclictest0-21swapper/321:45:533
229843399151,8cyclictest0-21swapper/321:45:533
229843399151,8cyclictest0-21swapper/321:15:533
229843399151,8cyclictest0-21swapper/320:10:573
229843399151,8cyclictest0-21swapper/300:05:563
229843399151,7cyclictest0-21swapper/323:00:523
229843399151,7cyclictest0-21swapper/322:35:523
229843399151,7cyclictest0-21swapper/321:00:503
229843399151,7cyclictest0-21swapper/320:50:553
229843399151,6cyclictest0-21swapper/319:25:503
229843399151,6cyclictest0-21swapper/319:25:503
229843399151,5cyclictest0-21swapper/320:30:483
229843399141,8cyclictest0-21swapper/323:05:433
229843399141,7cyclictest0-21swapper/323:45:523
229843399141,7cyclictest0-21swapper/323:40:393
229843399141,7cyclictest0-21swapper/323:30:553
229843399141,7cyclictest0-21swapper/323:15:523
229843399141,7cyclictest0-21swapper/322:45:513
229843399141,7cyclictest0-21swapper/322:40:493
229843399141,7cyclictest0-21swapper/322:30:503
229843399141,7cyclictest0-21swapper/322:15:503
229843399141,7cyclictest0-21swapper/322:15:503
229843399141,7cyclictest0-21swapper/320:56:003
229843399141,7cyclictest0-21swapper/320:56:003
229843399141,7cyclictest0-21swapper/320:40:493
229843399141,6cyclictest0-21swapper/319:35:503
229843399141,6cyclictest0-21swapper/319:30:503
229843399133,7cyclictest0-21swapper/321:40:573
229843399132,7cyclictest0-21swapper/322:50:503
229843399132,6cyclictest0-21swapper/323:10:493
229843399132,6cyclictest0-21swapper/321:10:453
229843399131,8cyclictest0-21swapper/300:00:533
229843399131,8cyclictest0-21swapper/300:00:533
229843399131,7cyclictest0-21swapper/321:50:513
229843399131,7cyclictest0-21swapper/321:30:473
229843399131,7cyclictest0-21swapper/321:20:493
229843399131,7cyclictest0-21swapper/300:20:563
229843399131,7cyclictest0-21swapper/300:15:523
229843399131,6cyclictest0-21swapper/320:00:493
229843399131,6cyclictest0-21swapper/319:55:493
229843399131,6cyclictest0-21swapper/319:15:573
229843399131,5cyclictest0-21swapper/319:45:483
229843399122,7cyclictest0-21swapper/300:35:443
229843399122,3cyclictest0-21swapper/319:10:593
229843399121,7cyclictest0-21swapper/323:35:393
229843399121,7cyclictest0-21swapper/322:25:383
229843399121,7cyclictest0-21swapper/322:05:523
229843399121,7cyclictest0-21swapper/322:00:443
229843399121,7cyclictest0-21swapper/320:05:493
229843399121,6cyclictest0-21swapper/321:55:523
229843399121,5cyclictest0-21swapper/321:05:533
229843399120,6cyclictest0-21swapper/320:45:533
229843399112,6cyclictest0-21swapper/321:25:413
229843399111,7cyclictest0-21swapper/322:55:413
229843399111,6cyclictest0-21swapper/320:35:473
229843399111,6cyclictest0-21swapper/319:55:013
229843399111,6cyclictest0-21swapper/319:20:503
229843399111,6cyclictest0-21swapper/300:25:403
229843399111,5cyclictest0-21swapper/320:15:533
229843399101,5cyclictest0-21swapper/320:20:493
22984339991,6cyclictest0-21swapper/319:10:023
22984339981,5cyclictest0-21swapper/319:40:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional