You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-04 - 00:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Jun 03, 2026 12:42:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
11092996736,17cyclictest12350irq/32-eth010:37:000
11092995240,8cyclictest12350irq/32-eth010:01:010
11094994831,3cyclictest171rcu_preempt10:01:072
1109299483,38cyclictest152627-21sh11:56:010
1109299483,38cyclictest152627-21sh11:56:010
11092994814,27cyclictest57788-21ssh09:26:030
11094994638,5cyclictest104075-21systemd-run10:40:012
11093994536,5cyclictest81905-21munin-plugin-st10:05:011
11093994536,5cyclictest26093-21idleruntime-cro08:00:011
11094994335,5cyclictest113681-21latency_hist10:55:022
11093994336,4cyclictest167247-21basename12:20:011
11093994336,4cyclictest0-21swapper/110:26:101
11093994336,4cyclictest0-21swapper/110:26:101
11093994335,6cyclictest0-21swapper/110:20:011
11093994335,6cyclictest0-21swapper/110:20:011
11093994335,5cyclictest14114-21systemd-run07:20:011
11092994234,5cyclictest45606-21systemd-run09:05:010
11093994132,6cyclictest3408135-21cups-browsed11:12:341
11093994116,15cyclictest271ktimers/109:36:041
11093994114,22cyclictest118885-21kworker/u8:4+events_unbound11:21:001
11092994132,6cyclictest115202-21timerwakeupswit10:56:130
11092994131,7cyclictest420-21avahi-daemon11:01:000
11093994032,5cyclictest47152-21latency_hist09:10:011
11093994032,5cyclictest47152-21latency_hist09:10:011
11092994026,11cyclictest0-21swapper/009:31:140
1109299402,10cyclictest0-21swapper/009:41:060
11094993930,5cyclictest17103-21systemd-run07:30:012
1109399393,25cyclictest92600-21ssh10:21:021
1109399393,25cyclictest92600-21ssh10:21:021
11092993930,6cyclictest9876-21kworker/u8:12-xprtiod07:43:150
1109499384,27cyclictest177969-21ssh12:36:062
11094993830,5cyclictest420-21avahi-daemon09:20:572
11094993830,5cyclictest135671-21systemd-run11:30:012
1109499382,27cyclictest143316-21ssh11:41:082
11093993830,5cyclictest1-21systemd11:45:021
11093993828,7cyclictest47076-21diskmemload11:05:591
11092993832,4cyclictest47906-21fschecks_count09:10:550
11092993832,4cyclictest47906-21fschecks_count09:10:550
11094993730,4cyclictest98546-21fschecks_time10:30:592
11094993730,4cyclictest98546-21fschecks_time10:30:592
11094993729,5cyclictest28690-21mailstats08:06:112
11093993729,4cyclictest0-21swapper/110:36:011
11092993728,6cyclictest46113-21fschecks_time09:05:560
11092993728,6cyclictest46113-21fschecks_time09:05:560
11092993727,4cyclictest123015-21cut11:10:010
11092993727,4cyclictest0-21swapper/011:36:010
1109299371,30cyclictest165321-21ssh12:16:030
11094993630,3cyclictest0-21swapper/209:46:072
11094993629,4cyclictest141963-21sendmail11:40:012
11094993628,5cyclictest116671-21ssh11:00:002
11093993630,3cyclictest0-21swapper/111:05:011
11093993628,5cyclictest29602-21fschecks_count08:10:561
11093993627,6cyclictest140571-21tail11:36:141
1109399362,27cyclictest158962-21ssh12:06:011
1109299368,21cyclictest12350irq/32-eth011:46:060
11092993630,3cyclictest3529357-21kworker/u8:6-xprtiod07:40:010
11092993626,7cyclictest39573-21cron08:45:000
11092993626,7cyclictest39573-21cron08:45:000
11092993624,8cyclictest86614-21switchtime10:11:130
1109499353,24cyclictest130574-21ssh11:21:032
11094993530,2cyclictest0-21swapper/210:15:012
11094993527,5cyclictest153166-21sed11:56:142
11094993527,5cyclictest153166-21sed11:56:142
11094993526,6cyclictest548-21gdbus08:26:022
11093993527,5cyclictest0-21swapper/108:40:001
11093993527,5cyclictest0-21swapper/108:40:001
11093993527,4cyclictest0-21swapper/110:41:021
11093993527,4cyclictest0-21swapper/109:16:091
11093993526,6cyclictest58380-21sed09:26:161
11092993523,3cyclictest171rcu_preempt12:11:060
11092993521,10cyclictest12350irq/32-eth011:11:100
11092993518,11cyclictest12350irq/32-eth008:01:060
11094993426,5cyclictest69230-21sh09:45:012
11094993426,5cyclictest47076-21diskmemload12:11:022
11094993426,5cyclictest157759-21cron12:05:002
11094993426,4cyclictest9876-21kworker/u8:12-rpciod07:43:122
11094993413,11cyclictest351ktimers/210:06:062
11093993428,3cyclictest0-21swapper/111:27:111
11093993426,5cyclictest516-21in:imuxsock09:50:011
11093993423,8cyclictest127154-21latency11:15:571
11093993422,5cyclictest3408135-21cups-browsed10:14:221
11092993427,5cyclictest24618-21latency_hist07:55:020
11092993427,3cyclictest0-21swapper/011:51:000
1109299342,25cyclictest83093-21ssh10:06:010
11092993417,14cyclictest127929-21unixbench_multi11:16:130
11092993411,15cyclictest12350irq/32-eth012:36:090
11094993327,4cyclictest11091-21cyclictest11:30:562
11094993326,4cyclictest167230-21munin-run12:20:012
11094993325,6cyclictest351ktimers/208:31:032
11094993325,6cyclictest351ktimers/208:31:032
11094993325,5cyclictest170432-21systemd-run12:25:012
11094993324,6cyclictest159459-21timerwakeupswit12:06:132
1109499331,26cyclictest171763-21ssh12:26:092
11093993328,3cyclictest0-21swapper/112:00:011
11093993328,3cyclictest0-21swapper/112:00:011
11093993326,4cyclictest174376-21fschecks_count12:30:571
11093993326,3cyclictest0-21swapper/111:46:071
11093993325,5cyclictest78806-21latency_hist10:00:021
11093993325,5cyclictest170393-21ssh12:25:011
11093993325,5cyclictest110403-21latency_hist10:50:011
11093993325,5cyclictest0-21swapper/111:51:141
11093993322,8cyclictest18589-21sh07:35:011
1109399332,22cyclictest114813-21ssh10:56:041
1109399331,26cyclictest67306-21ssh09:41:031
1109299339,6cyclictest171rcu_preempt08:26:060
1109299331,24cyclictest133937-21sshd11:26:110
1109499324,22cyclictest105327-21ssh10:41:042
1109499323,21cyclictest76879-21ssh09:56:072
11094993226,3cyclictest0-21swapper/208:20:012
11094993225,4cyclictest91368-21sh10:20:012
11094993225,4cyclictest91368-21sh10:20:012
11094993224,5cyclictest176845-21cat12:35:022
11094993220,5cyclictest1017-21sendmail-mta11:05:232
11093993226,3cyclictest0-21swapper/108:20:011
11093993225,3cyclictest0-21swapper/110:06:061
11093993224,5cyclictest56593-21systemd-run09:25:011
1109299323,23cyclictest159104-21ssh12:06:080
11092993226,4cyclictest91360-21cron10:20:010
11092993226,4cyclictest91360-21cron10:20:010
11092993226,4cyclictest137266-21timerandwakeup11:31:160
11092993226,3cyclictest0-21swapper/008:11:090
11092993224,5cyclictest421-21dbus-daemon09:00:020
11092993224,5cyclictest3408135-21cups-browsed08:30:590
11092993224,5cyclictest3408135-21cups-browsed08:30:590
11094993125,2cyclictest0-21swapper/207:25:012
1109499312,22cyclictest127355-21rm11:16:012
11094993121,7cyclictest62898-21cut09:35:012
11093993126,2cyclictest0-21swapper/110:55:001
11093993121,8cyclictest50275-21ssh09:15:011
11093993121,8cyclictest50275-21ssh09:15:011
11092993124,3cyclictest0-21swapper/009:21:020
11092993121,7cyclictest70365-21needreboot09:45:590
11092993117,11cyclictest0-21swapper/012:20:530
11094993024,3cyclictest0-21swapper/209:55:012
11094993023,4cyclictest117535-21fschecks_count11:00:562
11094993022,5cyclictest23096-21kworker/u8:1+xprtiod08:00:012
11094993022,5cyclictest13105-21fschecks_time07:15:552
1109499302,21cyclictest149632-21ssh11:51:082
1109499302,20cyclictest108557-21ssh10:46:072
11094993021,6cyclictest3408135-21cups-browsed08:56:092
11094993020,7cyclictest51397-21needreboot09:16:022
11093993025,2cyclictest0-21swapper/108:05:011
11093993025,2cyclictest0-21swapper/107:25:011
11093993022,5cyclictest43959-21tail08:56:141
11092993025,3cyclictest42180-21mailstats08:51:120
1109299302,21cyclictest73577-21ssh09:51:000
1109499296,13cyclictest1055-21mta-sts-daemon07:51:092
11094992922,3cyclictest0-21swapper/210:25:522
11094992922,3cyclictest0-21swapper/210:25:522
11094992921,5cyclictest421-21dbus-daemon07:10:022
11094992920,7cyclictest19344-21missed_timers07:35:592
1109399298,5cyclictest47076-21diskmemload12:15:011
11093992922,4cyclictest0-21swapper/109:05:011
1109399292,20cyclictest99029-21ssh10:31:121
1109399292,20cyclictest99029-21ssh10:31:121
11093992921,5cyclictest3408135-21cups-browsed08:06:351
11093992920,5cyclictest0-21swapper/112:26:091
11093992915,8cyclictest171rcu_preempt12:01:011
1109399291,20cyclictest73630-21ssh09:51:021
11092992924,4cyclictest0-21swapper/009:16:100
11092992923,4cyclictest41097-21systemd-run08:50:010
11092992923,4cyclictest113513-21ssh10:55:000
11092992922,5cyclictest421-21dbus-daemon07:21:110
11092992922,4cyclictest91405-21kworker/u8:10-rpciod11:41:080
11092992922,4cyclictest130305-21kernelversion11:20:570
11092992919,7cyclictest108513-21ntp_offset10:46:050
11092992918,9cyclictest0-21swapper/010:30:010
11092992918,9cyclictest0-21swapper/010:30:010
11092992911,15cyclictest12350irq/32-eth009:36:020
1109499283,20cyclictest659-21snmpd08:41:022
1109499283,20cyclictest659-21snmpd08:41:022
11094992821,4cyclictest45575-21cron09:05:002
11094992821,3cyclictest0-21swapper/211:45:572
11094992820,5cyclictest42470-21unixbench_multi08:51:152
11094992813,11cyclictest421-21dbus-daemon08:21:062
11093992821,5cyclictest17044-21unixbench_singl07:26:151
11093992820,5cyclictest40119-21fschecks_time08:45:561
11092992822,3cyclictest1054-21mta-sts-daemon07:11:060
11092992814,9cyclictest12350irq/32-eth008:06:030
1109299281,23cyclictest92637-21ssh10:21:040
1109299281,23cyclictest92637-21ssh10:21:040
11094992721,3cyclictest0-21swapper/209:11:102
11094992721,3cyclictest0-21swapper/209:11:102
11094992720,5cyclictest64663-21grep09:36:132
11094992720,3cyclictest0-21swapper/209:26:022
11094992718,6cyclictest1-21systemd08:40:012
11094992718,6cyclictest1-21systemd08:40:012
11093992722,3cyclictest0-21swapper/112:35:551
11093992721,3cyclictest0-21swapper/108:55:001
11093992721,3cyclictest0-21swapper/107:55:011
11093992720,4cyclictest32625-21sort08:20:571
11093992720,4cyclictest11598-21fschecks_count07:10:561
11093992719,5cyclictest3408135-21cups-browsed08:29:531
1109299277,13cyclictest12350irq/32-eth012:31:060
11092992719,5cyclictest439-21systemd-logind07:30:010
11092992715,8cyclictest0-21swapper/009:56:020
11092992711,10cyclictest105290-21ssh10:41:030
11094992621,2cyclictest0-21swapper/210:25:002
11094992621,2cyclictest0-21swapper/210:25:002
11094992621,2cyclictest0-21swapper/207:31:142
11094992619,4cyclictest427-21polkitd07:46:102
11094992617,5cyclictest124264-21ntpq11:11:052
11094992615,7cyclictest41016-21unixbench_singl08:46:142
1109399262,4cyclictest171rcu_preempt07:36:071
1109399262,18cyclictest136913-21ntpq11:31:071
11093992621,2cyclictest0-21swapper/107:43:161
11093992619,4cyclictest22116-21fschecks_time07:45:561
1109299265,11cyclictest47076-21diskmemload10:31:140
1109299265,11cyclictest47076-21diskmemload10:31:140
11092992620,4cyclictest26087-21sendmail08:00:010
11092992619,5cyclictest3408135-21cups-browsed07:47:160
11092992618,6cyclictest38104-21sh08:40:010
11092992618,6cyclictest38104-21sh08:40:010
11092992617,7cyclictest32762-21kernelversion08:20:590
11093992516,3cyclictest1051-21mta-sts-daemon09:34:231
11093992513,10cyclictest36520-21unixbench_singl08:31:151
11093992513,10cyclictest36520-21unixbench_singl08:31:151
11092992519,4cyclictest17608-21grep07:30:560
1109499243,16cyclictest3408135-21cups-browsed08:01:062
11094992418,3cyclictest0-21swapper/209:10:002
11094992418,3cyclictest0-21swapper/209:10:002
11094992417,4cyclictest30563-21cron08:15:002
11092992419,3cyclictest155783-21kworker/u8:2-rpciod12:26:060
11094992319,2cyclictest0-21swapper/207:11:142
1109299232,15cyclictest155834-21ssh12:01:030
11092992314,7cyclictest13389-21needreboot07:16:000
11093992213,6cyclictest38606-21fschecks_count08:40:551
11093992213,6cyclictest38606-21fschecks_count08:40:551
1109299222,15cyclictest31538-21ntpq08:16:070
11093991913,4cyclictest11191-21latency_hist07:10:021
1109599182,8cyclictest0-21swapper/311:11:073
1109599182,7cyclictest0-21swapper/309:50:513
1109599178,4cyclictest0-21swapper/310:36:013
1109599173,7cyclictest0-21swapper/311:26:063
1109599173,6cyclictest0-21swapper/308:06:033
1109599173,4cyclictest0-21swapper/308:31:023
1109599173,4cyclictest0-21swapper/308:31:023
1109299171,13cyclictest11047-21munin-run07:10:010
1109599164,4cyclictest0-21swapper/307:41:043
1109599162,8cyclictest0-21swapper/311:51:013
1109599162,8cyclictest0-21swapper/311:41:083
1109599162,8cyclictest0-21swapper/310:06:013
1109599161,7cyclictest0-21swapper/307:16:053
1109599154,5cyclictest0-21swapper/311:21:003
1109599153,8cyclictest0-21swapper/310:56:023
1109599152,9cyclictest0-21swapper/312:37:113
1109599152,8cyclictest0-21swapper/311:05:553
1109599152,8cyclictest0-21swapper/309:46:073
1109599152,8cyclictest0-21swapper/309:16:063
1109599152,7cyclictest0-21swapper/309:56:023
1109599152,6cyclictest0-21swapper/310:25:563
1109599152,6cyclictest0-21swapper/310:25:563
1109599151,8cyclictest0-21swapper/312:11:073
1109599151,8cyclictest0-21swapper/311:31:113
1109599151,8cyclictest0-21swapper/310:16:023
1109599151,8cyclictest0-21swapper/310:16:023
1109599151,8cyclictest0-21swapper/310:15:013
1109599151,8cyclictest0-21swapper/310:01:053
1109599151,8cyclictest0-21swapper/309:00:553
1109599151,6cyclictest0-21swapper/308:56:013
1109599151,6cyclictest0-21swapper/307:46:043
1109599143,7cyclictest0-21swapper/311:56:043
1109599143,7cyclictest0-21swapper/311:56:043
1109599143,5cyclictest0-21swapper/312:20:513
1109599143,5cyclictest0-21swapper/310:46:003
1109599142,7cyclictest0-21swapper/312:15:593
1109599142,7cyclictest0-21swapper/310:41:013
1109599142,7cyclictest0-21swapper/309:35:573
1109599142,6cyclictest0-21swapper/312:01:023
1109599142,6cyclictest0-21swapper/311:35:533
1109599142,6cyclictest0-21swapper/310:21:013
1109599142,6cyclictest0-21swapper/310:21:013
1109599141,8cyclictest0-21swapper/312:31:113
1109599141,7cyclictest0-21swapper/312:26:083
1109599141,6cyclictest0-21swapper/307:31:023
1109599141,6cyclictest0-21swapper/307:11:033
1109599140,6cyclictest0-21swapper/307:56:033
1109599133,6cyclictest0-21swapper/310:30:533
1109599133,6cyclictest0-21swapper/310:30:533
1109599132,7cyclictest0-21swapper/311:50:023
1109599132,6cyclictest0-21swapper/308:16:023
1109599132,4cyclictest0-21swapper/312:06:053
1109599131,8cyclictest0-21swapper/309:11:053
1109599131,8cyclictest0-21swapper/309:11:053
1109599131,7cyclictest0-21swapper/311:16:003
1109599131,7cyclictest0-21swapper/310:51:013
1109599131,7cyclictest0-21swapper/309:41:033
1109599131,7cyclictest0-21swapper/309:31:013
1109599131,7cyclictest0-21swapper/309:20:543
1109599131,7cyclictest0-21swapper/308:26:043
1109599126,4cyclictest0-21swapper/308:25:013
1109599122,6cyclictest0-21swapper/311:00:523
1109599122,6cyclictest0-21swapper/309:25:573
1109599122,5cyclictest0-21swapper/307:51:083
1109599121,8cyclictest0-21swapper/308:15:023
1109599121,6cyclictest0-21swapper/309:06:093
1109599121,6cyclictest0-21swapper/309:06:093
1109599121,6cyclictest0-21swapper/308:36:093
1109599121,6cyclictest0-21swapper/308:36:093
1109599121,5cyclictest0-21swapper/308:46:013
1109599101,6cyclictest0-21swapper/308:01:013
1109599101,6cyclictest0-21swapper/307:25:563
110959993,4cyclictest421rcuc/307:10:013
110959992,4cyclictest0-21swapper/308:41:043
110959992,4cyclictest0-21swapper/308:41:043
110959991,5cyclictest0-21swapper/307:35:533
110959991,4cyclictest0-21swapper/307:25:033
110959980,5cyclictest0-21swapper/308:51:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional