You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 20:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Fri May 15, 2026 12:43:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
797280995843,10cyclictest4149711-21cups-browsed10:30:581
797280994839,6cyclictest991520-21aten_r2power_fr12:25:411
797279994835,10cyclictest912804-21ssh10:47:340
797280994739,5cyclictest1051-21mta-sts-daemon11:55:551
797279994638,6cyclictest4149711-21cups-browsed11:36:060
797281994436,5cyclictest889870-21grep10:20:012
797280994435,5cyclictest0-21swapper/110:47:341
797279994437,4cyclictest889869-21munin-run10:20:010
797279994436,5cyclictest938343-21systemd-run11:20:010
797281994334,6cyclictest986819-21systemd-run12:20:012
797281994333,6cyclictest1-21systemd07:30:012
797280994335,5cyclictest833286-21ssh09:10:011
797279994335,5cyclictest998947-21systemd-run12:35:010
797281994234,5cyclictest837332-21systemd-run09:15:012
79728199422,35cyclictest846795-21ssh09:25:542
797280994233,6cyclictest824259-21basename08:40:011
797281994133,5cyclictest906128-21latency_hist10:40:022
797281994132,6cyclictest421-21dbus-daemon07:15:592
797281994032,5cyclictest930269-21latency_hist11:10:022
797281994032,5cyclictest910068-21systemd-run10:45:012
797281994031,6cyclictest936246-21unixbench_singl11:16:062
797280994033,5cyclictest0-21swapper/111:50:011
797280994032,5cyclictest931738-21proc_pri11:10:591
797280994032,5cyclictest931738-21proc_pri11:10:591
797280994032,5cyclictest930256-21systemd-run11:10:021
797279994033,5cyclictest899867-21unixbench_multi10:31:040
797281993932,4cyclictest0-21swapper/210:47:342
797280993933,3cyclictest0-21swapper/110:40:021
797280993931,5cyclictest834161-21fschecks_count09:10:451
797280993931,5cyclictest806789-21fschecks_count07:40:451
797280993929,7cyclictest4149711-21cups-browsed11:02:001
797280993929,6cyclictest420-21avahi-daemon09:39:141
797279993931,5cyclictest4149711-21cups-browsed07:20:010
797279993923,14cyclictest0-21swapper/008:25:010
79727999391,30cyclictest883157-21ssh10:10:560
797280993829,6cyclictest420-21avahi-daemon08:20:021
797280993829,6cyclictest261-21systemd-journal09:17:281
797280993827,8cyclictest806257-21munin-run07:40:011
797279993826,10cyclictest862324-21aten_r2power_po09:45:430
79727999381,32cyclictest895376-21ssh10:25:570
79727999381,32cyclictest895376-21ssh10:25:570
797281993729,5cyclictest1002978-21munin-run12:40:012
797281993729,5cyclictest1002978-21munin-run12:40:012
797280993731,3cyclictest0-21swapper/111:20:011
797280993730,4cyclictest867569-21tail09:51:061
797280993729,5cyclictest907249-21needreboot10:40:521
797280993729,5cyclictest858891-21ssh09:40:521
797280993728,6cyclictest4149711-21cups-browsed09:00:431
797280993728,6cyclictest4149711-21cups-browsed09:00:431
797280993726,8cyclictest850682-21needreboot09:30:491
797279993730,4cyclictest857784-21latency_hist09:40:030
797281993628,5cyclictest4149711-21cups-browsed11:14:502
797281993628,5cyclictest4149711-21cups-browsed11:14:502
797281993627,5cyclictest990841-21munin-plugin-st12:25:012
797281993626,7cyclictest1055-21mta-sts-daemon10:55:512
79728199362,24cyclictest979980-21ssh12:10:522
797280993630,4cyclictest830255-21sendmail09:00:001
797280993628,5cyclictest548-21gdbus07:20:011
797280993625,8cyclictest833254-21diskmemload12:25:011
797279993629,3cyclictest922461-21kworker/u8:5-events_unbound11:05:570
797279993628,5cyclictest421-21dbus-daemon12:30:010
797281993528,3cyclictest0-21swapper/210:20:582
797281993526,6cyclictest819784-21true08:25:012
797281993524,8cyclictest940134-21timerwakeupswit11:21:042
797280993527,5cyclictest919012-21fschecks_time10:55:451
797280993527,5cyclictest889888-21idleruntime-cro10:20:011
797280993527,5cyclictest877810-21systemd10:05:021
797280993527,5cyclictest1002983-21ssh12:40:011
797280993527,5cyclictest1002983-21ssh12:40:011
797279993529,4cyclictest833254-21diskmemload09:26:050
797279993526,5cyclictest982892-21latency_hist12:15:020
797281993428,3cyclictest0-21swapper/209:55:012
797280993428,3cyclictest0-21swapper/109:25:011
797280993427,5cyclictest797278-21cyclictest12:05:451
797280993427,4cyclictest871176-21proc_pri09:55:591
797280993426,5cyclictest952171-21switchtime11:36:051
797280993426,5cyclictest885963-21latency_hist10:15:031
797280993425,6cyclictest420-21avahi-daemon12:17:031
797279993429,2cyclictest0-21swapper/012:10:010
797279993428,4cyclictest999800-21ssh12:35:430
797279993428,4cyclictest999800-21ssh12:35:430
797279993425,7cyclictest954731-21kworker/u8:6-rpciod11:50:010
797281993328,2cyclictest0-21swapper/209:20:012
797281993328,2cyclictest0-21swapper/208:55:012
797281993327,3cyclictest0-21swapper/211:30:012
797281993326,4cyclictest0-21swapper/210:00:452
797281993326,3cyclictest0-21swapper/212:25:432
797281993325,5cyclictest548-21gdbus09:56:002
797281993325,5cyclictest1-21systemd12:10:022
797281993324,6cyclictest4149711-21cups-browsed10:50:162
797281993323,7cyclictest440-21thd12:00:572
797281993323,5cyclictest0-21swapper/211:55:572
797281993322,8cyclictest899033-21latency10:30:472
797280993325,5cyclictest822745-21cron08:35:011
797280993323,7cyclictest938961-21basename11:20:421
797279993328,2cyclictest0-21swapper/009:25:010
797279993327,4cyclictest955819-21ssh11:40:530
797279993327,4cyclictest923463-21ntp_kernel_pll_11:00:520
797279993326,5cyclictest0-21swapper/009:40:420
797279993326,4cyclictest948383-21unixbench_singl11:31:050
797279993325,5cyclictest922203-21wc11:00:020
797279993325,5cyclictest0-21swapper/010:06:040
797281993224,5cyclictest830171-21head08:56:022
797281993224,5cyclictest548-21gdbus11:46:012
797281993224,3cyclictest0-21swapper/209:35:012
797281993221,8cyclictest813162-21ntp_kernel_pll_08:00:532
797280993227,2cyclictest0-21swapper/109:25:421
797280993226,3cyclictest0-21swapper/112:15:021
797280993226,3cyclictest0-21swapper/111:40:561
797280993225,5cyclictest894517-21/usr/sbin/munin10:25:411
797280993225,5cyclictest894517-21/usr/sbin/munin10:25:411
797280993225,4cyclictest1056-21mta-sts-daemon11:51:121
797280993225,4cyclictest1056-21mta-sts-daemon11:51:121
797280993224,5cyclictest797339-21latency_hist07:10:021
797280993222,8cyclictest812282-21sh08:00:011
797279993227,2cyclictest0-21swapper/010:36:090
797279993223,6cyclictest978726-21kworker/u8:2-xprtiod12:21:050
797279993221,8cyclictest0-21swapper/008:00:000
797281993126,3cyclictest0-21swapper/208:20:012
797281993124,4cyclictest801767-21cron07:25:012
797281993124,4cyclictest1055-21mta-sts-daemon10:06:572
797281993123,5cyclictest842300-21fschecks_time09:20:472
797281993123,5cyclictest798782-21systemd-run07:15:012
797281993122,5cyclictest4149711-21cups-browsed11:43:382
797281993121,7cyclictest863290-21switchtime09:46:032
797280993124,4cyclictest881910-21latency_hist10:10:021
797279993125,3cyclictest0-21swapper/011:55:190
797279993125,3cyclictest0-21swapper/008:35:010
797279993124,5cyclictest4149711-21cups-browsed09:17:360
797279993124,4cyclictest798776-21systemd-run07:15:010
797279993122,7cyclictest802672-21ntp_kernel_pll_07:25:530
797281993025,2cyclictest0-21swapper/211:40:012
797281993024,3cyclictest0-21swapper/212:35:012
797281993024,3cyclictest0-21swapper/210:15:012
797281993024,3cyclictest0-21swapper/208:30:012
797281993024,2cyclictest0-21swapper/211:30:552
797281993023,3cyclictest0-21swapper/211:50:512
797281993023,3cyclictest0-21swapper/211:50:512
797281993022,3cyclictest0-21swapper/209:41:282
797281993021,5cyclictest810756-21munin-plugin-st07:55:002
797280993024,3cyclictest0-21swapper/112:30:571
797280993024,3cyclictest0-21swapper/111:30:511
797280993024,2cyclictest0-21swapper/108:45:531
797280993022,5cyclictest825288-21proc_pri08:40:591
797280993021,6cyclictest420-21avahi-daemon10:53:331
797280993021,5cyclictest819684-21unixbench_singl08:21:031
79727999304,24cyclictest833254-21diskmemload12:05:010
79727999304,15cyclictest151ktimers/007:20:510
797279993024,4cyclictest1051-21mta-sts-daemon10:50:530
797279993023,4cyclictest934391-21latency_hist11:15:030
797279993023,4cyclictest934391-21latency_hist11:15:030
797279993022,5cyclictest867429-21timerandwakeup09:51:050
797279993020,4cyclictest891053-21missed_timers10:20:480
797281992919,7cyclictest807084-21needreboot07:40:492
797280992923,3cyclictest0-21swapper/112:05:011
797280992922,3cyclictest0-21swapper/110:20:501
797280992921,5cyclictest801766-21cron07:25:011
797280992920,7cyclictest828422-21switchtime08:51:021
797280992920,7cyclictest816469-21timerandwakeup08:11:011
797280992920,7cyclictest808667-21ntp_kernel_pll_07:45:541
797280992920,7cyclictest808667-21ntp_kernel_pll_07:45:541
797280992920,7cyclictest803269-21/usr/sbin/munin07:30:021
797279992923,3cyclictest0-21swapper/007:40:010
797279992922,3cyclictest0-21swapper/009:30:570
79728199283,21cyclictest428-21mta-sts-daemon08:05:552
797281992821,4cyclictest857554-21cron09:40:012
797281992820,5cyclictest826126-21aten_r2power_po08:45:442
797281992820,5cyclictest4149711-21cups-browsed08:30:502
797281992818,7cyclictest824024-21timerwakeupswit08:36:032
797281992817,5cyclictest0-21swapper/210:25:592
797281992817,5cyclictest0-21swapper/210:25:592
797281992810,3cyclictest171rcu_preempt11:00:442
797280992823,2cyclictest0-21swapper/109:46:051
797280992820,5cyclictest1055-21mta-sts-daemon07:30:481
797280992817,8cyclictest943019-21basename11:25:431
797279992823,3cyclictest872907-21kworker/u8:12-writeback11:20:520
797279992821,5cyclictest151ktimers/009:00:530
797279992821,5cyclictest151ktimers/009:00:530
797279992821,4cyclictest832286-21fschecks_time09:05:470
797279992821,4cyclictest829257-21fschecks_count08:55:440
797279992820,5cyclictest983942-21missed_timers12:15:510
797281992720,5cyclictest1054-21mta-sts-daemon08:10:532
797281992720,4cyclictest832292-21fschecks_time09:05:472
797281992719,5cyclictest811265-21fschecks_count07:55:462
797280992720,4cyclictest807794-21kworker/u8:5-xprtiod08:00:471
797280992719,5cyclictest798630-21unixbench_multi07:11:051
797279992721,4cyclictest946398-21cron11:30:010
797279992720,5cyclictest0-21swapper/007:45:510
797279992720,5cyclictest0-21swapper/007:45:510
797279992720,4cyclictest827169-21head08:46:040
797279992720,4cyclictest816749-21sh08:15:010
797279992717,8cyclictest0-21swapper/011:50:460
797279992717,8cyclictest0-21swapper/011:50:460
79727999271,20cyclictest151ktimers/008:25:570
79727999264,12cyclictest833254-21diskmemload10:44:200
797279992618,6cyclictest875203-21ntpq10:01:010
797279992616,7cyclictest825134-21ntp_kernel_pll_08:40:520
797279992520,3cyclictest870592-21grep09:55:460
797279992519,4cyclictest814290-21fschecks_time08:05:450
797279992517,6cyclictest1055-21mta-sts-daemon09:13:060
79728199242,17cyclictest831165-21ntpq09:00:542
79728199242,17cyclictest831165-21ntpq09:00:542
797281992416,6cyclictest806320-21latency_hist07:40:022
79728099242,16cyclictest814745-21ntpq08:05:571
797279992418,3cyclictest0-21swapper/007:55:000
79728299232,12cyclictest0-21swapper/310:25:533
79728299232,12cyclictest0-21swapper/310:25:533
797281992318,3cyclictest1056-21mta-sts-daemon07:33:252
79728099232,16cyclictest820629-21ntpq08:25:521
797279992317,3cyclictest0-21swapper/008:36:040
797279992315,6cyclictest803935-21latency07:30:490
79728299224,8cyclictest0-21swapper/310:00:533
79728299224,10cyclictest0-21swapper/312:05:593
79728299223,12cyclictest0-21swapper/309:30:503
79728299223,10cyclictest0-21swapper/311:20:583
79728299222,12cyclictest0-21swapper/309:20:583
79728299222,11cyclictest0-21swapper/310:47:343
79728199222,15cyclictest825219-21ntpq08:40:572
797280992215,4cyclictest0-21swapper/107:51:021
79727999222,14cyclictest817707-21ntpq08:15:570
79728299213,11cyclictest0-21swapper/310:55:573
79728299213,11cyclictest0-21swapper/310:35:553
79728299213,10cyclictest0-21swapper/311:30:533
79728299212,12cyclictest0-21swapper/310:20:583
79728299212,10cyclictest0-21swapper/309:35:573
79728299212,10cyclictest0-21swapper/309:15:583
797281992114,3cyclictest351ktimers/207:45:572
797281992114,3cyclictest351ktimers/207:45:572
79727999212,14cyclictest828101-21ntpq08:50:510
79727999212,14cyclictest813145-21ntpq08:00:520
79727999211,15cyclictest807294-21ps07:40:580
79728299204,10cyclictest0-21swapper/309:40:513
79728299203,11cyclictest0-21swapper/310:30:583
79728299203,11cyclictest0-21swapper/308:20:533
79728299203,10cyclictest0-21swapper/311:45:513
79728299202,9cyclictest0-21swapper/311:25:523
79728299202,12cyclictest0-21swapper/310:50:443
79728299202,11cyclictest0-21swapper/312:16:013
79728299202,11cyclictest0-21swapper/307:25:543
79728299202,10cyclictest0-21swapper/312:10:523
79728299202,10cyclictest0-21swapper/311:55:553
79728299202,10cyclictest0-21swapper/310:40:563
79728299193,9cyclictest0-21swapper/309:10:523
79728299193,11cyclictest0-21swapper/309:55:513
79728299192,12cyclictest0-21swapper/312:20:543
79728299192,11cyclictest0-21swapper/311:10:533
79728299192,11cyclictest0-21swapper/311:10:533
79728299192,10cyclictest0-21swapper/312:25:533
79728299192,10cyclictest0-21swapper/311:05:573
79728299191,11cyclictest0-21swapper/311:40:563
797279991915,2cyclictest0-21swapper/007:10:010
79728299183,8cyclictest0-21swapper/312:35:493
79728299183,8cyclictest0-21swapper/312:35:493
79728299183,10cyclictest0-21swapper/309:45:563
79728299182,8cyclictest0-21swapper/308:50:563
79728299182,11cyclictest0-21swapper/310:05:523
79728299182,11cyclictest0-21swapper/309:50:503
79728299182,10cyclictest0-21swapper/312:00:573
79728299182,10cyclictest0-21swapper/311:35:553
79728299182,10cyclictest0-21swapper/311:15:543
79728299182,10cyclictest0-21swapper/309:25:543
79728299181,9cyclictest0-21swapper/312:30:503
79728299181,9cyclictest0-21swapper/307:45:573
79728299181,9cyclictest0-21swapper/307:45:573
797281991812,3cyclictest0-21swapper/207:10:012
79728299174,7cyclictest0-21swapper/311:50:443
79728299174,7cyclictest0-21swapper/311:50:443
79728299173,8cyclictest0-21swapper/310:10:543
79728299172,8cyclictest0-21swapper/308:20:013
79728299172,7cyclictest0-21swapper/308:45:513
79728299172,7cyclictest0-21swapper/308:05:513
79728299172,10cyclictest0-21swapper/310:15:503
79728299171,8cyclictest0-21swapper/308:30:013
79728299171,6cyclictest0-21swapper/309:00:513
79728299171,6cyclictest0-21swapper/309:00:513
79728299164,5cyclictest0-21swapper/308:40:503
79728299162,9cyclictest0-21swapper/311:00:463
79728299162,7cyclictest0-21swapper/307:30:543
79728299153,7cyclictest0-21swapper/308:30:583
79728299152,6cyclictest0-21swapper/307:35:513
79728299151,8cyclictest0-21swapper/309:10:013
79728299151,8cyclictest0-21swapper/307:40:553
79728299151,7cyclictest0-21swapper/308:35:513
79728299142,6cyclictest0-21swapper/307:15:533
79728299141,8cyclictest0-21swapper/308:55:493
79728299141,8cyclictest0-21swapper/307:10:563
79728299141,7cyclictest0-21swapper/308:00:543
79728299141,7cyclictest0-21swapper/307:50:503
79728299141,6cyclictest0-21swapper/308:15:033
79728299141,6cyclictest0-21swapper/307:20:513
79728299131,6cyclictest0-21swapper/307:55:513
79728299122,8cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional