You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 06:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Fri Aug 28, 2026 00:42:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
37900995850,5cyclictest187124-21latency_hist00:15:022
37899995346,4cyclictest0-21swapper/100:38:401
3789899512,23cyclictest75228-21ssh21:11:140
37899995039,5cyclictest0-21swapper/123:51:051
37900994840,5cyclictest72407-21systemd-run21:05:012
37899994843,2cyclictest0-21swapper/119:40:001
37899994843,2cyclictest0-21swapper/119:40:001
3789899474,31cyclictest154657-21ssh23:21:060
37898994732,12cyclictest125902-21idleruntime-cro22:35:010
37898994529,2cyclictest0-21swapper/021:31:040
37900994436,5cyclictest76918-21cron21:15:012
37899994436,5cyclictest180959-21systemd-run00:05:021
37900994334,6cyclictest116765-21systemd-run22:20:012
3789999432,36cyclictest111911-21ssh22:11:061
37898994337,4cyclictest46918-21systemd-run19:40:010
37898994337,4cyclictest46918-21systemd-run19:40:010
37898994328,11cyclictest40935-21latency_hist19:20:020
37900994234,5cyclictest694-21cron22:55:012
37900994132,6cyclictest2389223-21cups-browsed22:20:342
37899994135,3cyclictest0-21swapper/100:00:011
37899994133,5cyclictest52857-21cron20:00:011
37899994131,6cyclictest41091-21latency_hist19:20:021
37899994131,4cyclictest0-21swapper/123:11:011
37900994033,4cyclictest1002-21mta-sts-daemon22:07:572
37900994032,4cyclictest54407-21kworker/u8:8-xprtiod20:20:012
37900994030,7cyclictest60383-21cron20:25:012
37899994032,5cyclictest135052-21cron22:50:011
37898994034,4cyclictest59416-21fschecks_count20:20:560
37900993933,3cyclictest0-21swapper/219:36:032
37900993933,3cyclictest0-21swapper/219:36:032
37900993931,5cyclictest157239-21cut23:25:552
37898993928,8cyclictest127720-21head22:36:180
37900993831,4cyclictest150315-21ssh23:15:012
37900993831,2cyclictest0-21swapper/220:00:012
37900993830,6cyclictest151245-21if_wlan023:15:592
37900993830,4cyclictest0-21swapper/200:38:372
37900993829,6cyclictest123375-21/usr/sbin/munin22:30:522
37899993829,6cyclictest1-21systemd21:30:011
37899993829,5cyclictest485-21in:imuxsock21:05:011
37899993829,5cyclictest265-21systemd-journal00:25:571
3789899388,5cyclictest37897-21cyclictest21:21:040
37898993824,3cyclictest151ktimers/023:26:040
37900993730,4cyclictest113678-21grep22:15:002
37900993730,4cyclictest1002-21mta-sts-daemon21:46:042
37900993729,4cyclictest1001-21mta-sts-daemon23:51:082
37900993728,4cyclictest0-21swapper/223:31:022
3790099372,26cyclictest665-21snmpd21:26:032
37899993730,5cyclictest187038-21ssh00:15:021
37899993728,5cyclictest77732-21grep21:15:581
3789999370,29cyclictest75135-21ssh21:11:091
3789899373,29cyclictest151637-21sshd23:16:080
37898993730,5cyclictest177899-21cups-browsed00:21:180
37898993730,5cyclictest177899-21cups-browsed00:21:180
37898993730,4cyclictest56069-21latency_hist20:10:030
37898993730,4cyclictest101438-21ssh21:55:010
3790099364,25cyclictest99684-21ssh21:51:072
37900993628,5cyclictest102522-21grep21:55:592
37900993628,5cyclictest102522-21grep21:55:592
37900993627,4cyclictest0-21swapper/220:40:012
37899993628,4cyclictest0-21swapper/121:31:091
37899993626,7cyclictest1002-21mta-sts-daemon20:15:021
3789999362,27cyclictest169943-21ssh23:46:061
37898993629,4cyclictest151ktimers/019:41:040
37898993627,5cyclictest151ktimers/020:46:030
3789899361,30cyclictest114966-21ssh22:16:060
37900993529,3cyclictest0-21swapper/221:31:142
37900993528,5cyclictest37897-21cyclictest22:42:522
37900993528,3cyclictest0-21swapper/221:21:022
37900993527,5cyclictest168792-21latency_hist23:45:022
37900993527,3cyclictest0-21swapper/223:00:012
37900993527,3cyclictest0-21swapper/221:16:062
37900993526,6cyclictest420-21avahi-daemon00:01:132
3789999353,24cyclictest665-21snmpd21:46:081
37899993528,4cyclictest69382-21kworker/u8:3-xprtiod22:36:171
37899993528,3cyclictest0-21swapper/121:36:051
37899993527,5cyclictest60416-21systemd-run20:25:021
37899993527,5cyclictest2389223-21cups-browsed22:28:061
37899993525,7cyclictest188732-21grep00:16:141
37899993525,7cyclictest188732-21grep00:16:141
37899993525,6cyclictest1002-21mta-sts-daemon23:34:541
37898993528,4cyclictest193966-21fschecks_count00:25:590
37898993528,4cyclictest138087-21ssh22:55:010
37898993524,8cyclictest43208-21needreboot19:25:590
37900993428,2cyclictest0-21swapper/223:40:012
37900993427,5cyclictest0-21swapper/200:30:532
37900993427,4cyclictest0-21swapper/200:20:012
37900993427,4cyclictest0-21swapper/200:20:012
37900993427,3cyclictest0-21swapper/221:36:112
37900993427,3cyclictest0-21swapper/221:10:012
37900993427,3cyclictest0-21swapper/219:55:012
37900993426,5cyclictest193668-21gdbus00:25:542
3790099342,27cyclictest182220-21ssh00:06:062
37899993427,4cyclictest141250-21latency_hist23:00:021
37899993426,5cyclictest73909-21systemd21:10:021
37899993426,5cyclictest42403-21cut19:25:011
37899993426,5cyclictest265-21systemd-journal19:15:011
37899993423,8cyclictest0-21swapper/123:10:021
37899993411,19cyclictest98441-21kworker/u8:15+xprtiod22:41:021
37898993429,3cyclictest90236-21latency21:35:590
37898993428,4cyclictest53402-21fschecks_count20:00:560
37898993426,5cyclictest2389223-21cups-browsed23:39:360
37898993421,10cyclictest201807-40mandb00:38:380
37900993326,3cyclictest0-21swapper/222:46:022
37900993325,5cyclictest485-21in:imuxsock23:05:012
37900993325,3cyclictest0-21swapper/222:40:012
37899993326,4cyclictest158251-21head23:26:141
37899993326,4cyclictest0-21swapper/123:41:001
37899993325,5cyclictest73879-21diskmemload22:20:541
37899993325,5cyclictest51397-21systemd-run19:55:021
37899993323,6cyclictest153552-21latency_hist23:20:021
37898993325,6cyclictest58185-21needreboot20:16:000
37898993325,6cyclictest144233-21munin-plugin-st23:05:010
37898993324,6cyclictest167379-21tail23:41:120
37900993225,4cyclictest1000-21mta-sts-daemon23:46:292
37900993224,5cyclictest439-21systemd-logind00:25:012
37900993224,5cyclictest439-21systemd-logind00:25:012
37900993221,8cyclictest53793-21ntp_kernel_pll_20:01:042
37900993218,9cyclictest567-21gdbus19:26:032
3789999323,23cyclictest114853-21ssh22:16:021
37899993223,3cyclictest0-21swapper/121:21:051
37899993221,8cyclictest64116-21wc20:36:001
37898993227,3cyclictest1002-21mta-sts-daemon21:16:060
37898993225,4cyclictest185358-21grep00:11:110
37898993224,6cyclictest190038-21ssh00:20:000
37898993224,6cyclictest190038-21ssh00:20:000
37898993224,6cyclictest107577-21ls22:05:000
37898993224,6cyclictest107577-21ls22:05:000
37898993224,5cyclictest151ktimers/020:26:010
37898993222,5cyclictest0-21swapper/023:56:070
3789899322,24cyclictest182169-21ssh00:06:040
37900993123,5cyclictest67873-21munin-run20:50:012
37900993123,5cyclictest56979-21systemctl20:11:102
37900993123,5cyclictest265-21systemd-journal00:00:012
3790099312,23cyclictest121025-21ssh22:26:072
37900993114,13cyclictest351ktimers/220:26:072
37899993124,4cyclictest102086-21aten_r2power_po21:55:531
37899993124,4cyclictest102086-21aten_r2power_po21:55:531
37898993125,4cyclictest122968-21latency_hist22:30:020
37898993123,6cyclictest162617-21date23:35:020
3789899312,23cyclictest148676-21ssh23:11:090
37898993111,16cyclictest665-21snmpd21:41:060
37900993023,4cyclictest93265-21kernelversion21:40:582
37900993023,4cyclictest1003-21mta-sts-daemon20:41:072
37900993023,4cyclictest1001-21mta-sts-daemon23:06:362
37900993021,6cyclictest265-21systemd-journal21:00:012
37900993021,5cyclictest63304-21unixbench_singl20:31:142
37899993024,3cyclictest0-21swapper/121:41:051
37899993024,3cyclictest0-21swapper/120:51:031
37899993023,4cyclictest49896-21systemd-run19:50:021
37899993023,4cyclictest110016-21ssh22:08:081
37899993023,3cyclictest0-21swapper/119:41:081
37899993022,5cyclictest177899-21cups-browsed00:32:431
37899993022,4cyclictest53222-21aten_r2power_fr20:00:531
3789999302,22cyclictest142518-21ssh23:01:091
37899993021,6cyclictest45286-21unixbench_multi19:31:201
37899993021,6cyclictest45286-21unixbench_multi19:31:201
37899993021,6cyclictest136565-21/usr/sbin/munin22:51:131
37898993026,2cyclictest0-21swapper/022:21:110
37898993024,3cyclictest1002-21mta-sts-daemon22:48:490
37898993024,3cyclictest0-21swapper/021:56:050
37898993024,3cyclictest0-21swapper/021:56:050
37898993023,4cyclictest2389223-21cups-browsed22:57:480
37898993021,7cyclictest50687-21needreboot19:50:580
37898993017,10cyclictest64960-21latency_hist20:40:020
37900992924,2cyclictest0-21swapper/219:21:122
37900992922,3cyclictest0-21swapper/223:21:052
37900992921,5cyclictest0-21swapper/222:01:002
37900992921,5cyclictest0-21swapper/222:01:002
37899992922,4cyclictest1003-21mta-sts-daemon00:08:441
37899992921,5cyclictest66324-21unixbench_singl20:41:141
37899992921,5cyclictest1-21systemd00:25:011
37899992921,5cyclictest1-21systemd00:25:011
37899992921,4cyclictest0-21swapper/122:00:301
37899992921,4cyclictest0-21swapper/122:00:301
3789899293,19cyclictest12650irq/32-eth023:06:040
37898992923,4cyclictest68400-21fschecks_time20:50:550
37898992917,9cyclictest88880-21kworker/u8:2-nfsiod23:51:070
37900992820,4cyclictest0-21swapper/219:41:132
37900992819,6cyclictest1-21systemd19:15:012
37900992817,8cyclictest49258-21ntp_kernel_pll_19:46:022
37899992822,3cyclictest0-21swapper/123:21:041
37899992821,3cyclictest0-21swapper/120:46:101
37899992820,5cyclictest73879-21diskmemload22:35:011
37899992820,5cyclictest70883-21sendmail21:00:011
3789899283,17cyclictest96625-21ssh21:46:060
37898992821,4cyclictest265-21systemd-journal20:15:010
37898992814,9cyclictest151ktimers/021:06:060
37899992721,3cyclictest0-21swapper/121:51:111
37899992719,5cyclictest2389223-21cups-browsed20:32:421
37898992722,3cyclictest199271-21systemd00:35:010
37898992718,7cyclictest109067-21mailstats22:06:130
37899992618,6cyclictest421-21dbus-daemon19:25:521
37899992618,5cyclictest54851-21cpuspeed20:05:541
3789899269,11cyclictest12650irq/32-eth000:01:050
3789899264,14cyclictest627-21lldpd23:47:040
37898992621,2cyclictest0-21swapper/021:26:010
37898992620,4cyclictest52869-21munin-plugin-st20:00:010
37898992620,4cyclictest1000-21mta-sts-daemon22:41:280
37898992619,4cyclictest1002-21mta-sts-daemon19:46:060
37898992617,7cyclictest1-21systemd19:30:480
37898992617,7cyclictest1-21systemd19:30:480
37900992517,6cyclictest44222-21tail19:30:522
37900992517,6cyclictest44222-21tail19:30:522
37900992517,3cyclictest0-21swapper/219:20:012
37900992516,4cyclictest0-21swapper/220:06:052
37899992517,5cyclictest420-21avahi-daemon19:10:011
37899992517,5cyclictest0-21swapper/120:20:011
37899992510,12cyclictest271ktimers/120:26:041
37898992518,5cyclictest69383-21kworker/u8:4-rpciod21:05:010
3790199242,12cyclictest0-21swapper/321:41:063
3790199242,11cyclictest0-21swapper/300:21:063
3790199242,11cyclictest0-21swapper/300:21:063
3790199241,11cyclictest0-21swapper/321:26:033
37899992418,3cyclictest0-21swapper/123:35:541
3789899242,17cyclictest111802-21ssh22:11:020
3790099233,15cyclictest665-21snmpd20:51:002
3789899231,9cyclictest38690-21needreboot19:11:000
37898992317,4cyclictest69751-21aten_r2power_vo20:55:530
37898992316,3cyclictest0-21swapper/020:41:040
37898992312,8cyclictest62146-21aten_r2power_cu20:30:520
3790199222,11cyclictest0-21swapper/323:31:103
3790199213,9cyclictest0-21swapper/323:06:053
3789899219,9cyclictest421-21dbus-daemon19:10:020
3789899212,13cyclictest41917-21ps19:21:070
3790199201,9cyclictest0-21swapper/321:01:083
3790199201,8cyclictest0-21swapper/322:36:053
3790199201,7cyclictest0-21swapper/320:11:013
3790199193,9cyclictest0-21swapper/300:11:013
3790199193,8cyclictest0-21swapper/323:11:003
3790199192,9cyclictest0-21swapper/321:10:013
3790199192,8cyclictest0-21swapper/322:31:043
3790199192,10cyclictest0-21swapper/322:46:153
3790199192,10cyclictest0-21swapper/321:31:023
3790199192,10cyclictest0-21swapper/300:06:063
3790199191,10cyclictest0-21swapper/321:16:043
3790199183,9cyclictest0-21swapper/322:56:023
3790199183,10cyclictest0-21swapper/300:38:413
3790199182,8cyclictest0-21swapper/320:46:063
3790199181,9cyclictest0-21swapper/323:36:043
3790199181,9cyclictest0-21swapper/300:16:093
3790199181,9cyclictest0-21swapper/300:16:093
3790199181,10cyclictest0-21swapper/322:21:063
37900991812,3cyclictest0-21swapper/219:10:012
3790199173,7cyclictest0-21swapper/319:41:033
3790199172,9cyclictest0-21swapper/323:51:073
3790199172,9cyclictest0-21swapper/323:01:103
3790199172,9cyclictest0-21swapper/321:36:023
3790199171,9cyclictest0-21swapper/322:51:053
3790199171,8cyclictest0-21swapper/320:26:033
3790199171,7cyclictest0-21swapper/322:15:093
3790199171,11cyclictest0-21swapper/321:46:043
3790199171,10cyclictest0-21swapper/321:56:073
3790199171,10cyclictest0-21swapper/321:56:073
3790199171,10cyclictest0-21swapper/300:25:553
3790199162,9cyclictest0-21swapper/321:21:153
3790199162,8cyclictest0-21swapper/320:21:063
3790199162,7cyclictest0-21swapper/322:26:063
3790199161,9cyclictest0-21swapper/322:41:083
3790199161,8cyclictest0-21swapper/321:11:093
3790199161,8cyclictest0-21swapper/319:56:053
3790199161,8cyclictest0-21swapper/319:26:033
3790199161,7cyclictest0-21swapper/320:56:043
3790199161,7cyclictest0-21swapper/319:50:593
3790199153,7cyclictest0-21swapper/322:01:113
3790199153,7cyclictest0-21swapper/322:01:113
3790199152,8cyclictest0-21swapper/321:51:063
3790199152,6cyclictest0-21swapper/319:21:043
3790199151,9cyclictest0-21swapper/323:15:583
3790199151,8cyclictest0-21swapper/322:11:033
3790199151,8cyclictest0-21swapper/300:30:573
3790199151,8cyclictest0-21swapper/300:01:053
3790199151,7cyclictest0-21swapper/320:41:053
3790199151,7cyclictest0-21swapper/320:16:043
3790199151,7cyclictest0-21swapper/319:36:023
3790199151,7cyclictest0-21swapper/319:36:023
3790199151,6cyclictest0-21swapper/320:51:003
3790199151,6cyclictest0-21swapper/319:11:023
3790199142,9cyclictest0-21swapper/323:46:103
3790199142,8cyclictest0-21swapper/322:06:043
3790199142,7cyclictest0-21swapper/323:41:003
3790199142,7cyclictest0-21swapper/323:20:533
3790199142,6cyclictest0-21swapper/320:06:053
3790199141,9cyclictest0-21swapper/323:56:033
3790199132,7cyclictest0-21swapper/319:35:013
3790199132,7cyclictest0-21swapper/319:35:013
3790199132,6cyclictest0-21swapper/323:26:023
3790199132,5cyclictest0-21swapper/320:35:593
3790199131,7cyclictest0-21swapper/319:46:013
3790199131,6cyclictest0-21swapper/320:01:033
3790199121,5cyclictest0-21swapper/319:16:013
3790199110,6cyclictest0-21swapper/320:31:013
379019993,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional