You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-24 - 12:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 24, 2026 00:42:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
605137995345,5cyclictest638159-21systemd-run21:00:022
605137995143,5cyclictest620119-21basename20:00:012
605137994941,5cyclictest1-21systemd22:15:012
605137994940,5cyclictest686769-21systemd-run22:25:012
605136994927,18cyclictest0-21swapper/100:06:021
605137994739,5cyclictest747033-21cut00:05:012
605137994739,5cyclictest747033-21cut00:05:012
60513699474,8cyclictest651615-21ntpq21:26:031
605136994640,3cyclictest0-21swapper/120:00:011
605136994638,5cyclictest421-21dbus-daemon20:30:011
605135994539,4cyclictest671677-21systemd-run22:00:010
60513699441,19cyclictest654646-21ssh21:31:021
60513699441,12cyclictest684978-21ntpq22:21:051
605135994436,5cyclictest694211-21mailstats22:36:090
605135994436,5cyclictest694211-21mailstats22:36:090
605136994334,6cyclictest766548-21systemctl00:36:111
605136994334,6cyclictest766548-21systemctl00:36:111
60513699432,6cyclictest615036-21ntpq19:41:021
605136994319,20cyclictest726685-21uniq23:30:571
605136994319,20cyclictest726685-21uniq23:30:571
60513699431,12cyclictest606087-21ntpq19:11:051
60513699431,11cyclictest688778-21sshd22:26:531
605135994338,3cyclictest605134-21cyclictest00:05:530
605137994233,6cyclictest763015-21if_wlan000:31:002
605136994232,8cyclictest0-21swapper/123:56:031
60513699422,6cyclictest630089-21ntpq20:31:041
605136994224,15cyclictest638114-21munin-run21:00:011
60513699421,13cyclictest721169-21ntp_states23:21:071
60513699421,13cyclictest721169-21ntp_states23:21:071
605137994131,7cyclictest608124-21/usr/sbin/munin19:20:022
60513699419,4cyclictest0-21swapper/122:41:131
60513699418,2cyclictest0-21swapper/123:06:061
60513699413,7cyclictest640608-21ntpq21:06:061
60513699413,7cyclictest640608-21ntpq21:06:061
605136994129,7cyclictest0-21swapper/123:41:051
60513699412,11cyclictest641897-21diskstats21:10:531
60513699412,11cyclictest641897-21diskstats21:10:531
60513699411,6cyclictest621116-21ntpq20:01:051
60513699411,11cyclictest633007-21ntpq20:41:001
60513699411,11cyclictest631354-21memory20:35:571
605137994034,3cyclictest0-21swapper/223:40:022
605137994031,6cyclictest687538-21sort22:25:532
605136994024,12cyclictest0-21swapper/123:20:001
605136994024,12cyclictest0-21swapper/123:20:001
605136994019,18cyclictest0-21swapper/122:45:561
60513699401,15cyclictest657821-21ntpq21:36:061
60513699401,12cyclictest682936-21ssh22:17:311
60513699401,12cyclictest664738-21ssh21:47:121
60513699401,10cyclictest613484-21ntpq19:36:001
605135994032,5cyclictest1000-21mta-sts-daemon22:49:230
605135994021,16cyclictest12750irq/33-eth022:51:050
605137993931,5cyclictest737920-21cron23:50:012
605137993931,5cyclictest699581-21fschecks_time22:45:562
605137993930,6cyclictest704788-21ssh22:55:002
605137993930,5cyclictest659586-21cron21:40:012
60513699397,2cyclictest0-21swapper/119:25:011
60513699393,10cyclictest675846-21ssh22:06:011
60513699392,9cyclictest702933-21ssh22:50:581
605136993929,7cyclictest759118-21grep00:25:011
60513699392,6cyclictest634550-21ntpq20:46:031
60513699392,6cyclictest634550-21ntpq20:46:031
60513699392,6cyclictest623995-21ntpq20:10:591
60513699392,6cyclictest622546-21ntpq20:06:021
60513699391,9cyclictest627031-21ntpq20:21:011
60513699391,9cyclictest616583-21ntp_states19:46:041
60513699391,9cyclictest612095-21ntp_states19:31:051
60513699391,9cyclictest612095-21ntp_states19:31:051
60513699391,7cyclictest607519-21ntpq19:16:011
605136993916,20cyclictest0-21swapper/123:46:041
60513699391,15cyclictest648519-21ntpq21:21:021
60513699391,13cyclictest745246-21ntpq00:01:051
60513699391,13cyclictest745246-21ntpq00:01:051
60513699391,12cyclictest755953-21ssh00:19:391
60513699391,12cyclictest694068-21ntpq22:36:061
60513699391,12cyclictest694068-21ntpq22:36:061
60513699391,12cyclictest625591-21ntpq20:16:071
60513699391,11cyclictest752084-21ssh00:11:541
60513699391,11cyclictest691971-21ssh22:32:281
605135993932,4cyclictest734921-21cron23:45:010
605135993929,7cyclictest682362-21timerwakeupswit22:16:160
605135993928,8cyclictest666728-21needreboot21:51:010
605136993828,6cyclictest744038-21cups-browsed00:29:111
605136993819,16cyclictest1000-21mta-sts-daemon22:11:031
605136993817,18cyclictest0-21swapper/123:00:011
605136993812,22cyclictest730954-21sh23:37:011
60513699381,10cyclictest646444-21ssh21:17:281
605136993810,5cyclictest0-21swapper/123:50:591
605135993833,3cyclictest1002-21mta-sts-daemon21:36:010
605135993831,4cyclictest757629-21switchtime00:21:120
605135993831,4cyclictest756065-21sh00:20:010
605135993830,5cyclictest753047-21cron00:15:000
605137993729,5cyclictest635116-21cron20:50:002
605137993729,5cyclictest635116-21cron20:50:002
60513699377,7cyclictest763359-21awk00:31:111
605136993729,5cyclictest265-21systemd-journal19:10:011
60513699371,9cyclictest673525-21ssh22:01:201
60513699371,9cyclictest638995-21ntpq21:01:021
605136993711,23cyclictest671208-21sh21:58:401
605136993711,23cyclictest641129-21diskmemload23:00:561
60513699371,11cyclictest661534-21ssh21:41:351
60513699371,10cyclictest641129-21diskmemload23:11:201
605135993731,4cyclictest650364-21cron21:25:010
605135993731,3cyclictest0-21swapper/021:11:020
605135993731,3cyclictest0-21swapper/021:11:020
605135993730,5cyclictest3032758-21cups-browsed23:20:000
605135993730,5cyclictest3032758-21cups-browsed23:20:000
605135993728,6cyclictest421-21dbus-daemon22:32:410
605137993629,4cyclictest753073-21kworker/u8:13+xprtiod00:26:152
605137993612,20cyclictest627-21lldpd19:30:592
605137993612,20cyclictest627-21lldpd19:30:592
60513699363,9cyclictest428-21mta-sts-daemon20:51:071
60513699363,5cyclictest724084-21ssh23:26:031
60513699363,5cyclictest724084-21ssh23:26:031
60513699362,8cyclictest610524-21ntpq19:26:021
60513699361,10cyclictest666043-21ssh21:50:411
605135993623,11cyclictest624951-21aten2.4-expect20:15:520
605137993527,5cyclictest647277-21munin-plugin-st21:20:002
605137993526,6cyclictest420-21avahi-daemon22:17:482
605135993526,7cyclictest730434-21switchtime23:36:120
605135993526,7cyclictest726989-21needreboot23:31:010
605135993526,7cyclictest726989-21needreboot23:31:010
605137993426,5cyclictest742696-21unixbench_multi23:56:132
605137993424,7cyclictest766042-21kernelversion00:35:582
605137993424,7cyclictest766042-21kernelversion00:35:582
605135993429,3cyclictest0-21swapper/000:40:010
605135993429,3cyclictest0-21swapper/000:40:010
605135993428,4cyclictest623019-21tail20:06:120
605135993427,4cyclictest725904-21systemd-run23:30:010
605135993427,4cyclictest725904-21systemd-run23:30:010
605135993425,6cyclictest420-21avahi-daemon23:02:490
605135993422,6cyclictest722872-21ssh23:25:000
605135993422,6cyclictest722872-21ssh23:25:000
605135993421,2cyclictest0-21swapper/021:45:010
60513799337,15cyclictest421-21dbus-daemon21:11:022
60513799337,15cyclictest421-21dbus-daemon21:11:022
605137993325,5cyclictest751586-21switchtime00:11:132
605137993325,5cyclictest671627-21ssh22:00:002
605137993324,5cyclictest640167-21fschecks_time21:05:542
605137993324,5cyclictest640167-21fschecks_time21:05:542
60513799332,27cyclictest696990-21ssh22:41:062
60513799331,30cyclictest705973-21rm22:56:022
60513699334,7cyclictest567-21gdbus19:53:221
605135993328,2cyclictest0-21swapper/023:00:010
605135993327,4cyclictest711525-21cpu23:05:520
605135993327,4cyclictest672439-21sort22:00:530
605135993327,3cyclictest633741-21latency_hist20:45:020
605135993327,3cyclictest0-21swapper/021:20:010
605135993325,5cyclictest420-21avahi-daemon00:29:000
60513799324,23cyclictest654746-21ssh21:31:062
605137993226,3cyclictest0-21swapper/220:45:012
605137993225,4cyclictest647412-21kworker/u8:7-xprtiod22:40:002
605137993225,4cyclictest647412-21kworker/u8:7-xprtiod22:40:002
605137993225,4cyclictest1002-21mta-sts-daemon23:18:232
605137993225,4cyclictest1002-21mta-sts-daemon23:18:232
605137993224,5cyclictest747665-21aten2.4-expect00:05:522
605137993224,5cyclictest732448-21sh23:40:492
605137993224,5cyclictest714441-21aten_r2power_fr23:10:522
605137993223,6cyclictest651040-21aten_r2power_vo21:25:522
605137993223,6cyclictest421-21dbus-daemon20:51:012
605137993222,4cyclictest0-21swapper/223:21:022
605137993222,4cyclictest0-21swapper/223:21:022
60513599326,21cyclictest687961-21ssh22:26:010
605135993226,4cyclictest678295-21ssh22:10:510
605135993226,4cyclictest614059-21head19:36:120
605135993226,4cyclictest421-21dbus-daemon21:10:020
605135993226,4cyclictest421-21dbus-daemon21:10:020
605135993224,6cyclictest714441-21aten_r2power_fr23:10:520
605135993222,5cyclictest0-21swapper/021:35:010
60513599321,15cyclictest3032758-21cups-browsed20:46:040
60513599321,15cyclictest3032758-21cups-browsed20:46:040
60513799314,23cyclictest648629-21ssh21:21:072
605137993126,2cyclictest0-21swapper/220:25:012
605137993126,2cyclictest0-21swapper/219:25:012
605137993123,6cyclictest724333-21mailstats23:26:112
605137993123,6cyclictest724333-21mailstats23:26:112
605137993123,5cyclictest611144-21systemd-run19:30:012
605137993119,7cyclictest641129-21diskmemload23:51:032
605135993125,4cyclictest628163-21fschecks_time20:25:540
605135993125,3cyclictest0-21swapper/000:05:010
605135993125,3cyclictest0-21swapper/000:05:010
605135993122,7cyclictest630625-21/usr/sbin/munin20:35:020
605135993120,8cyclictest12650irq/32-eth023:51:020
605135993120,8cyclictest0-21swapper/023:50:010
605137993025,2cyclictest0-21swapper/222:06:092
605137993024,4cyclictest108130-21kworker/u8:11-xprtiod19:40:002
605137993023,4cyclictest1001-21mta-sts-daemon23:31:502
605137993023,4cyclictest1001-21mta-sts-daemon23:31:502
605137993022,5cyclictest0-21swapper/220:30:012
605137993017,10cyclictest673080-21mailstats22:01:072
605135993023,5cyclictest615559-21head19:41:120
605135993023,4cyclictest744050-21cat00:00:020
605135993016,10cyclictest265-21systemd-journal21:27:520
60513799292,8cyclictest691054-21ntpq22:31:052
605137992922,3cyclictest0-21swapper/200:16:072
60513799292,21cyclictest660846-21ssh21:41:022
605137992921,5cyclictest625134-21fschecks_count20:15:562
605137992921,5cyclictest618642-21systemd19:55:022
605137992921,5cyclictest615630-21cron19:45:002
605137992920,6cyclictest566-21gmain21:47:492
605137992920,6cyclictest3032758-21cups-browsed23:05:122
605137992918,8cyclictest708314-21acpi23:00:502
60513599294,20cyclictest1003-21mta-sts-daemon21:01:020
605135992923,4cyclictest675717-21missed_timers22:05:580
605135992923,3cyclictest0-21swapper/000:31:220
605135992922,5cyclictest1-21systemd21:50:020
605135992921,6cyclictest694-21cron20:50:020
605135992920,6cyclictest420-21avahi-daemon20:21:050
60513799282,19cyclictest666766-21ntpq21:51:032
605137992821,3cyclictest0-21swapper/220:40:012
605137992819,6cyclictest421-21dbus-daemon20:30:582
605135992821,4cyclictest1000-21mta-sts-daemon22:23:350
60513599281,22cyclictest606146-21ps19:11:080
605137992722,3cyclictest1001-21mta-sts-daemon20:05:582
605137992721,3cyclictest0-21swapper/221:05:002
605137992719,5cyclictest621553-21unixbench_singl20:01:112
605137992718,5cyclictest1-21systemd20:10:532
605135992722,2cyclictest0-21swapper/019:15:540
605135992720,4cyclictest1002-21mta-sts-daemon19:53:010
605137992620,3cyclictest722994-21kworker/u8:5-xprtiod00:20:552
605137992620,3cyclictest0-21swapper/219:46:032
605135992621,3cyclictest1003-21mta-sts-daemon20:35:500
605135992620,3cyclictest0-21swapper/022:42:510
605135992612,5cyclictest12750irq/33-eth020:00:530
605135992516,6cyclictest624612-21cron20:15:010
605137992418,3cyclictest0-21swapper/219:15:012
60513599246,15cyclictest616412-21sed19:45:570
605135992416,5cyclictest1-21systemd19:35:010
605135992416,5cyclictest1-21systemd19:35:010
605135992415,7cyclictest637461-21ntp_kernel_err20:55:590
60513599233,7cyclictest608994-21ntpq19:21:020
60513899223,9cyclictest0-21swapper/300:16:073
60513899223,12cyclictest0-21swapper/322:01:043
60513599227,12cyclictest610007-21awk19:25:520
60513899212,10cyclictest0-21swapper/322:16:043
605135992115,4cyclictest618951-21aten2.4-expect19:55:520
60513899202,9cyclictest0-21swapper/320:10:583
60513899202,12cyclictest0-21swapper/323:41:053
60513899194,9cyclictest0-21swapper/323:31:063
60513899194,9cyclictest0-21swapper/323:31:063
60513899194,9cyclictest0-21swapper/322:40:003
60513899194,9cyclictest0-21swapper/322:40:003
60513899193,9cyclictest0-21swapper/323:26:093
60513899193,9cyclictest0-21swapper/323:26:093
60513899193,11cyclictest0-21swapper/321:11:023
60513899193,11cyclictest0-21swapper/321:11:023
60513899193,11cyclictest0-21swapper/300:06:013
60513899193,10cyclictest0-21swapper/322:00:013
60513899193,10cyclictest0-21swapper/321:51:063
60513899192,9cyclictest0-21swapper/322:21:023
60513899192,9cyclictest0-21swapper/321:16:023
60513899192,11cyclictest0-21swapper/323:01:013
60513899192,11cyclictest0-21swapper/322:45:573
60513899192,11cyclictest0-21swapper/322:25:583
60513899192,11cyclictest0-21swapper/321:41:043
60513899191,10cyclictest0-21swapper/323:21:043
60513899191,10cyclictest0-21swapper/323:21:043
60513899191,10cyclictest0-21swapper/322:40:583
60513899185,9cyclictest0-21swapper/320:36:043
60513899184,9cyclictest0-21swapper/323:46:053
60513899184,9cyclictest0-21swapper/322:56:093
60513899182,9cyclictest0-21swapper/322:10:553
60513899182,11cyclictest0-21swapper/322:31:043
60513899182,10cyclictest0-21swapper/323:11:053
60513899182,10cyclictest0-21swapper/321:36:043
60513899182,10cyclictest0-21swapper/300:31:093
60513899182,10cyclictest0-21swapper/300:26:043
60513899181,11cyclictest0-21swapper/323:16:063
60513899181,11cyclictest0-21swapper/323:16:063
60513899181,11cyclictest0-21swapper/321:26:023
60513899181,10cyclictest0-21swapper/323:36:003
60513899181,10cyclictest0-21swapper/300:21:053
60513899173,7cyclictest0-21swapper/320:40:533
60513899173,7cyclictest0-21swapper/319:51:003
60513899172,9cyclictest0-21swapper/320:26:033
60513899172,9cyclictest0-21swapper/320:06:033
60513899172,9cyclictest0-21swapper/300:35:543
60513899172,9cyclictest0-21swapper/300:35:543
60513899172,9cyclictest0-21swapper/300:11:013
60513899172,8cyclictest0-21swapper/320:46:063
60513899172,8cyclictest0-21swapper/320:46:063
60513899172,8cyclictest0-21swapper/319:56:033
60513899172,7cyclictest0-21swapper/319:31:063
60513899172,7cyclictest0-21swapper/319:31:063
60513899172,11cyclictest0-21swapper/323:50:593
60513899172,10cyclictest0-21swapper/322:06:073
60513899172,10cyclictest0-21swapper/321:30:513
60513899171,9cyclictest0-21swapper/320:31:053
60513899171,11cyclictest0-21swapper/322:50:573
60513899171,10cyclictest0-21swapper/321:21:063
605137991710,4cyclictest605084-21latency_hist19:10:022
60513899162,9cyclictest0-21swapper/323:06:073
60513899162,8cyclictest0-21swapper/300:00:523
60513899162,8cyclictest0-21swapper/300:00:523
60513899162,7cyclictest0-21swapper/320:21:013
60513899162,7cyclictest0-21swapper/319:36:033
60513899161,8cyclictest0-21swapper/319:21:053
60513599161,12cyclictest605148-21munin-run19:10:010
60513899155,6cyclictest0-21swapper/319:11:013
60513899152,9cyclictest0-21swapper/323:55:583
60513899152,8cyclictest0-21swapper/321:46:023
60513899152,8cyclictest0-21swapper/320:16:043
60513899152,7cyclictest0-21swapper/321:10:013
60513899152,7cyclictest0-21swapper/321:10:013
60513899152,7cyclictest0-21swapper/321:01:023
60513899152,7cyclictest0-21swapper/320:55:593
60513899143,7cyclictest0-21swapper/319:16:013
60513899143,6cyclictest0-21swapper/320:55:013
60513899142,7cyclictest0-21swapper/319:41:003
60513899142,6cyclictest0-21swapper/319:26:033
60513899141,8cyclictest0-21swapper/319:45:533
60513899131,6cyclictest0-21swapper/320:00:553
6051389992,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional