You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 17:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Jul 15, 2026 12:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2355267995140,7cyclictest2504744-21mailstats12:15:492
2355266994939,5cyclictest1559552-21cups-browsed07:30:331
2355266994739,5cyclictest2389777-21systemd-run09:05:021
2355267994432,9cyclictest428-21mta-sts-daemon12:10:302
2355266994436,3cyclictest0-21swapper/109:40:041
2355265994431,10cyclictest2412603-21cron09:45:010
2355266994335,5cyclictest2420028-21sh09:55:521
2355267994234,5cyclictest2459726-21ssh11:00:542
2355266994235,4cyclictest2473144-21ssh11:25:001
2355266994235,4cyclictest1001-21mta-sts-daemon11:44:401
2355266994235,3cyclictest0-21swapper/109:10:431
2355266994234,6cyclictest2489539-21uname11:50:471
2355266994136,3cyclictest0-21swapper/112:20:451
2355266994136,3cyclictest0-21swapper/112:20:451
2355266994133,5cyclictest2497396-21systemd-run12:05:011
2355266994130,8cyclictest2374749-21idleruntime-cro08:15:011
2355265994135,3cyclictest1000-21mta-sts-daemon10:10:520
2355265994135,3cyclictest1000-21mta-sts-daemon10:10:520
2355267994033,4cyclictest2412629-21kworker/u8:12-rpciod10:25:012
235526799392,30cyclictest2498483-21ssh12:05:432
2355266993932,4cyclictest1003-21mta-sts-daemon07:15:401
2355266993931,5cyclictest2437503-21fschecks_count10:25:361
2355266993930,6cyclictest1000-21mta-sts-daemon12:05:511
2355266993928,7cyclictest2510966-21timerandwakeup12:25:541
2355266993928,7cyclictest2510966-21timerandwakeup12:25:541
2355267993832,3cyclictest0-21swapper/212:40:012
2355267993831,4cyclictest2473168-21idleruntime-cro11:25:012
2355267993830,5cyclictest421-21dbus-daemon11:35:012
2355267993829,6cyclictest2417273-21unixbench_multi09:50:552
2355266993830,5cyclictest2486859-21unixbench-2d11:45:531
2355266993830,5cyclictest2400427-21munin-plugin-st09:25:011
2355266993830,5cyclictest2400427-21munin-plugin-st09:25:011
2355266993830,4cyclictest2383776-21kworker/u8:2-rpciod09:50:581
2355266993830,4cyclictest1001-21mta-sts-daemon09:49:141
2355266993829,7cyclictest2459644-21unixbench_multi11:00:521
2355265993832,3cyclictest0-21swapper/008:30:010
2355265993830,5cyclictest2425605-21latency10:05:380
2355267993729,4cyclictest0-21swapper/209:20:482
2355267993729,4cyclictest0-21swapper/209:20:482
2355266993731,3cyclictest0-21swapper/112:00:011
2355266993729,5cyclictest2409548-21sendmail09:40:011
2355266993729,5cyclictest2406499-21idleruntime-cro09:35:001
2355265993712,17cyclictest151ktimers/008:30:380
235526599371,16cyclictest2352524-21kworker/u8:10+rpciod07:40:390
235526799367,24cyclictest2440938-21ssh10:30:412
2355267993628,5cyclictest2480313-21ssh11:35:442
2355267993628,5cyclictest2425140-21aten_r2power_cu10:05:322
2355267993628,5cyclictest2389826-21latency_hist09:05:022
2355267993628,5cyclictest2376249-21munin-plugin-st08:20:012
2355267993628,5cyclictest2368684-21unixbench_singl07:50:562
2355266993630,3cyclictest0-21swapper/110:40:471
2355266993630,3cyclictest0-21swapper/110:35:421
2355266993629,4cyclictest2506453-21munin-run12:20:011
2355266993629,4cyclictest2470100-21ssh11:20:001
2355266993628,5cyclictest2464179-21latency_hist11:10:031
2355266993628,5cyclictest2391250-21munin-run09:10:011
2355266993628,4cyclictest0-21swapper/110:15:491
2355265993630,4cyclictest2500422-21unin-run12:10:020
2355265993627,7cyclictest2397346-21sh09:20:010
2355265993626,7cyclictest2358274-21sendmail07:20:010
2355267993529,3cyclictest0-21swapper/207:15:012
2355267993527,5cyclictest2422386-21fschecks_time10:00:342
2355266993528,4cyclictest2376455-21latency_hist08:20:021
2355266993527,5cyclictest694-21cron11:30:001
2355266993527,5cyclictest2451950-21grep10:50:011
2355266993527,5cyclictest2425698-21missed_timers10:05:391
2355266993527,5cyclictest2397403-21systemd09:20:021
2355265993529,4cyclictest2473231-21latency_hist11:25:020
2355265993529,3cyclictest0-21swapper/011:00:430
2355265993528,4cyclictest2503410-21ssh12:15:010
2355265993528,3cyclictest0-21swapper/009:25:350
2355265993528,3cyclictest0-21swapper/009:25:350
2355265993522,11cyclictest2409548-21sendmail09:40:010
2355267993427,4cyclictest2421673-21munin-run10:00:012
2355266993426,5cyclictest1559552-21cups-browsed07:10:451
2355265993428,4cyclictest2442936-21latency_hist10:35:020
2355265993428,4cyclictest2421886-21latency_hist10:00:030
2355265993428,4cyclictest2371752-21cron08:05:010
2355267993326,4cyclictest2491384-21latency_hist11:55:022
2355267993326,4cyclictest2406478-21kworker/u8:3-nfsiod09:40:012
2355267993325,5cyclictest2395933-21unixbench_multi09:15:532
2355267993324,6cyclictest2439849-21systemd-run10:30:012
2355266993326,4cyclictest2359751-21cron07:25:011
2355266993326,3cyclictest0-21swapper/110:50:371
2355266993324,6cyclictest2455459-21head10:55:331
235526599337,16cyclictest151ktimers/011:45:440
2355265993327,4cyclictest2482292-21latency_hist11:40:020
2355265993323,7cyclictest2462722-21ssh11:05:560
2355265993322,8cyclictest2399019-21grep09:20:540
2355265993322,8cyclictest2399019-21grep09:20:540
2355265993320,7cyclictest12650irq/32-eth012:15:430
235526599331,25cyclictest2492376-21ssh11:55:420
235526599331,24cyclictest2392452-21ssh09:10:460
2355267993224,4cyclictest0-21swapper/208:55:002
2355267993223,5cyclictest2513176-21grep12:30:332
2355267993222,5cyclictest0-21swapper/209:25:432
2355267993222,5cyclictest0-21swapper/209:25:432
2355267993220,8cyclictest2508005-21timerwakeupswit12:20:542
2355267993220,8cyclictest2508005-21timerwakeupswit12:20:542
2355266993225,4cyclictest1002-21mta-sts-daemon12:30:141
2355266993225,3cyclictest0-21swapper/110:20:441
2355266993225,3cyclictest0-21swapper/107:40:011
2355266993224,4cyclictest0-21swapper/110:10:351
2355266993224,4cyclictest0-21swapper/110:10:351
2355265993222,6cyclictest2443270-21basename10:35:310
2355267993125,3cyclictest0-21swapper/211:05:462
2355267993124,4cyclictest1002-21mta-sts-daemon08:45:012
2355267993124,4cyclictest1002-21mta-sts-daemon08:45:012
2355267993124,3cyclictest0-21swapper/211:15:012
2355267993123,5cyclictest1559552-21cups-browsed12:29:402
2355267993123,5cyclictest1559552-21cups-browsed12:29:402
2355267993121,8cyclictest2486093-21tr11:45:372
235526699313,23cyclictest2422774-21ssh10:00:411
2355266993126,2cyclictest0-21swapper/108:40:011
2355266993123,5cyclictest2367129-21unixbench_multi07:45:521
2355266993123,4cyclictest1003-21mta-sts-daemon10:32:141
2355266993123,4cyclictest0-21swapper/112:10:321
2355266993122,7cyclictest2377834-21latency_hist08:25:021
2355266993121,7cyclictest2515886-21acpi12:35:321
2355266993119,9cyclictest2378598-21ntp_kernel_err08:25:391
235526699311,27cyclictest2477260-21ssh11:30:441
2355265993124,5cyclictest1559552-21cups-browsed07:50:550
2355265993124,4cyclictest2434363-21aten_r2power_vo10:20:330
2355265993123,5cyclictest485-21in:imuxsock07:30:010
2355265993123,5cyclictest485-21in:imuxsock07:30:010
2355265993123,5cyclictest1559552-21cups-browsed10:45:060
2355265993121,5cyclictest2467359-21ssh11:15:120
2355267993023,5cyclictest0-21swapper/211:20:012
2355267993022,5cyclictest2374767-21systemd-run08:15:012
235526799302,23cyclictest2492474-21ssh11:55:452
2355266993024,3cyclictest0-21swapper/109:25:501
2355266993024,3cyclictest0-21swapper/109:25:501
2355266993023,4cyclictest2482222-21munin-run11:40:001
2355266993023,4cyclictest2370182-21unixbench_singl07:55:551
235526599303,12cyclictest2391248-21diskmemload11:11:450
2355265993024,4cyclictest2388222-21cron09:00:010
2355265993024,4cyclictest2370279-21systemd-run08:00:010
2355265993022,6cyclictest694-21cron11:45:000
2355265993022,6cyclictest0-21swapper/009:10:020
2355265993021,6cyclictest2477783-21unixbench-2d11:30:550
2355267992922,4cyclictest2430753-21cron10:15:012
2355267992922,4cyclictest2430753-21cron10:15:012
2355267992922,3cyclictest0-21swapper/210:55:372
2355266992921,5cyclictest2355270-21systemd-run07:10:011
2355266992920,6cyclictest750-21ntpd08:05:531
2355266992919,7cyclictest2433825-21kworker/u8:14+nfsiod11:11:101
235526599293,12cyclictest2391248-21diskmemload09:54:520
2355265992924,3cyclictest0-21swapper/010:55:370
2355265992923,4cyclictest2377363-21systemctl08:20:530
2355265992922,5cyclictest2404553-21ntp_kernel_pll_09:30:410
2355265992922,5cyclictest2373145-21unixbench_multi08:05:540
2355265992919,7cyclictest2491462-21latency_hist11:55:020
2355265992915,4cyclictest12750irq/33-eth012:35:470
235526599291,22cyclictest2495534-21ssh12:00:470
2355267992820,5cyclictest2370244-21idleruntime-cro08:00:012
2355267992820,5cyclictest2360282-21fschecks_time07:25:332
2355267992820,5cyclictest2360282-21fschecks_time07:25:332
2355267992819,6cyclictest2443319-21ssh10:35:322
2355267992818,6cyclictest1000-21mta-sts-daemon11:44:592
2355267992817,8cyclictest1559552-21cups-browsed08:35:422
235526799281,15cyclictest2413822-21ssh09:45:482
235526699285,13cyclictest1559552-21cups-browsed08:40:511
235526699285,13cyclictest1559552-21cups-browsed08:40:511
2355266992823,3cyclictest0-21swapper/107:45:011
2355266992820,5cyclictest421-21dbus-daemon08:35:011
2355265992822,4cyclictest1559552-21cups-browsed09:01:260
2355265992822,3cyclictest0-21swapper/012:30:330
2355265992821,5cyclictest1559552-21cups-browsed10:52:040
2355265992821,4cyclictest1559552-21cups-browsed07:37:100
2355265992820,6cyclictest2422686-21needreboot10:00:380
2355265992820,5cyclictest2437323-21ssh10:25:330
2355267992721,3cyclictest1000-21mta-sts-daemon10:15:332
2355267992720,3cyclictest0-21swapper/209:10:372
2355267992717,6cyclictest2453467-21timerandwakeup10:50:542
2355265992721,4cyclictest2432195-21switchtime10:15:530
2355265992721,4cyclictest2356647-21unixbench_multi07:10:520
2355265992721,3cyclictest1001-21mta-sts-daemon11:25:270
235526599271,21cyclictest750-21ntpd12:28:210
235526599271,21cyclictest750-21ntpd12:28:210
235526599271,19cyclictest2413734-21ssh09:45:440
235526799264,3cyclictest171rcu_preempt09:30:412
2355267992620,3cyclictest1001-21mta-sts-daemon12:00:272
2355267992620,3cyclictest1001-21mta-sts-daemon08:45:442
2355267992620,3cyclictest1000-21mta-sts-daemon07:36:482
2355267992620,3cyclictest0-21swapper/209:45:012
2355267992619,4cyclictest1001-21mta-sts-daemon10:45:562
2355267992619,3cyclictest0-21swapper/208:05:492
2355267992618,6cyclictest1000-21mta-sts-daemon07:20:462
2355267992616,7cyclictest2446504-21cpu10:40:332
2355266992619,4cyclictest1001-21mta-sts-daemon08:45:451
2355266992619,3cyclictest0-21swapper/107:50:541
2355265992620,4cyclictest2384953-21threads08:45:520
2355265992620,4cyclictest1000-21mta-sts-daemon10:40:470
2355265992619,5cyclictest1000-21mta-sts-daemon12:20:400
2355265992619,5cyclictest1000-21mta-sts-daemon12:20:400
2355265992618,6cyclictest2367110-21unixbench_multi07:45:520
235526799254,16cyclictest2390766-21ps09:05:502
2355267992518,4cyclictest2366411-21kernelversion07:45:362
2355267992517,5cyclictest2379185-21unixbench_singl08:25:522
2355265992520,3cyclictest1001-21mta-sts-daemon08:50:520
2355265992518,4cyclictest2375266-21fschecks_time08:15:350
2355265992515,3cyclictest0-21swapper/007:30:350
235526799242,17cyclictest2474344-21ntpq11:25:452
2355267992419,2cyclictest0-21swapper/207:40:392
2355267992418,3cyclictest1002-21mta-sts-daemon08:20:402
2355267992416,6cyclictest2357280-21gltestperf07:15:352
2355267992415,6cyclictest2371314-21sendmail_mailqu08:00:522
2355266992414,6cyclictest2361266-21systemd-run07:30:011
2355266992414,6cyclictest2361266-21systemd-run07:30:011
2355265992412,3cyclictest0-21swapper/008:40:020
2355267992318,2cyclictest0-21swapper/209:00:012
2355267992317,3cyclictest0-21swapper/208:35:012
2355267992316,4cyclictest2353922-21kworker/u8:13+rpciod07:10:012
2355267992315,5cyclictest2362895-21latency_hist07:35:022
2355266992315,5cyclictest2387248-21fschecks_count08:55:341
2355266992314,6cyclictest2386618-21grep08:50:551
235526599231,17cyclictest2382554-21tail08:40:310
235526599231,17cyclictest2382554-21tail08:40:310
2355265992310,9cyclictest421-21dbus-daemon08:10:440
235526699222,15cyclictest2371215-21ntpq08:00:481
235526599201,14cyclictest2359188-21ntpq07:20:440
235526599171,13cyclictest2355270-21systemd-run07:10:010
235526899163,9cyclictest0-21swapper/308:40:513
235526899163,9cyclictest0-21swapper/308:40:513
235526899163,8cyclictest0-21swapper/310:20:513
235526899163,7cyclictest0-21swapper/311:45:453
235526899162,9cyclictest0-21swapper/310:45:013
235526899162,7cyclictest0-21swapper/311:00:443
235526899152,9cyclictest0-21swapper/312:35:443
235526899152,8cyclictest0-21swapper/312:30:393
235526899152,8cyclictest0-21swapper/311:59:593
235526899152,7cyclictest0-21swapper/311:50:423
235526899152,7cyclictest0-21swapper/311:35:443
235526899151,8cyclictest0-21swapper/309:40:053
235526899151,7cyclictest0-21swapper/310:05:443
235526899151,7cyclictest0-21swapper/307:20:473
235526899143,8cyclictest0-21swapper/312:25:463
235526899143,8cyclictest0-21swapper/312:25:463
235526899143,8cyclictest0-21swapper/310:00:363
235526899143,6cyclictest0-21swapper/310:50:473
235526899143,5cyclictest0-21swapper/307:15:443
235526899142,8cyclictest0-21swapper/312:05:403
235526899142,7cyclictest0-21swapper/311:25:013
235526899142,7cyclictest0-21swapper/307:10:433
235526899142,4cyclictest0-21swapper/307:45:393
235526899141,8cyclictest0-21swapper/311:30:433
235526899141,8cyclictest0-21swapper/310:25:423
235526899141,8cyclictest0-21swapper/309:35:443
235526899141,7cyclictest0-21swapper/310:55:373
235526899141,7cyclictest0-21swapper/310:30:393
235526899141,7cyclictest0-21swapper/310:10:403
235526899141,7cyclictest0-21swapper/310:10:403
235526899141,6cyclictest0-21swapper/308:35:443
235526899141,6cyclictest0-21swapper/307:55:423
235526899141,6cyclictest0-21swapper/307:40:413
235526899141,6cyclictest0-21swapper/307:35:403
235526899133,7cyclictest0-21swapper/311:05:353
235526899133,6cyclictest0-21swapper/309:29:083
235526899133,6cyclictest0-21swapper/309:29:083
235526899133,4cyclictest0-21swapper/307:30:423
235526899132,8cyclictest0-21swapper/311:40:423
235526899132,7cyclictest0-21swapper/312:15:313
235526899132,7cyclictest0-21swapper/309:17:573
235526899132,6cyclictest0-21swapper/311:25:443
235526899132,6cyclictest0-21swapper/308:10:443
235526899131,8cyclictest0-21swapper/310:35:353
235526899131,7cyclictest0-21swapper/312:10:343
235526899131,7cyclictest0-21swapper/309:50:493
235526899131,7cyclictest0-21swapper/309:10:393
235526899122,6cyclictest0-21swapper/311:15:303
235526899122,6cyclictest0-21swapper/310:15:383
235526899121,7cyclictest0-21swapper/312:00:343
235526899121,7cyclictest0-21swapper/310:45:313
235526899121,7cyclictest0-21swapper/309:55:413
235526899121,7cyclictest0-21swapper/309:45:323
235526899121,7cyclictest0-21swapper/309:30:293
235526899121,7cyclictest0-21swapper/309:20:443
235526899121,7cyclictest0-21swapper/309:20:443
235526899121,6cyclictest0-21swapper/308:20:383
235526899121,6cyclictest0-21swapper/308:05:013
235526899121,6cyclictest0-21swapper/307:55:023
235526899121,5cyclictest0-21swapper/309:05:423
235526899121,5cyclictest0-21swapper/309:00:403
235526899112,5cyclictest0-21swapper/308:50:423
235526899111,7cyclictest0-21swapper/312:20:463
235526899111,7cyclictest0-21swapper/312:20:463
235526899111,7cyclictest0-21swapper/311:10:373
235526899111,6cyclictest0-21swapper/308:55:423
235526899111,6cyclictest0-21swapper/307:25:333
235526899111,6cyclictest0-21swapper/307:25:333
235526899101,6cyclictest0-21swapper/308:30:323
235526899101,5cyclictest0-21swapper/308:50:023
235526899101,5cyclictest0-21swapper/308:25:353
235526899100,6cyclictest0-21swapper/308:10:023
23552689993,5cyclictest0-21swapper/308:15:473
23552689980,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional