You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 09:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Sep 24, 2026 00:42:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
281223995949,7cyclictest420-21avahi-daemon22:31:212
28122299584,14cyclictest352245-21ssh22:06:201
28122299584,14cyclictest352245-21ssh22:06:201
281222995737,16cyclictest408837-21ssh23:40:011
281222995341,7cyclictest0-21swapper/122:01:241
28122299523,8cyclictest324743-21ssh21:21:221
281222995226,22cyclictest365001-21unixbench_multi22:26:341
281221995042,5cyclictest1-21systemd22:55:020
281222994939,6cyclictest380265-21tail22:51:321
281223994635,8cyclictest294737-21sh19:55:012
281223994536,5cyclictest379034-21aten_r2power_en22:51:102
281222994538,5cyclictest281220-21cyclictest00:20:001
28122299451,14cyclictest292663-21ntpq19:46:211
281223994436,5cyclictest360173-21latency_hist22:20:022
28122299443,7cyclictest313639-21ntpq20:56:201
281221994427,10cyclictest151ktimers/022:56:300
281223994334,6cyclictest310248-21grep20:46:082
28122299434,8cyclictest298632-21ntpq20:06:211
28122299431,12cyclictest437748-21ntpq00:26:221
28122299431,10cyclictest301631-21ntpq20:16:191
281223994233,6cyclictest485-21in:imuxsock00:00:022
28122399422,33cyclictest388796-21ssh23:06:162
281222994234,5cyclictest421121-21ssh00:00:021
28122299422,6cyclictest289680-21ntpq19:36:211
28122299422,5cyclictest373675-21ntpq22:41:211
28122299422,18cyclictest425393-21ssh00:06:161
28122299422,18cyclictest425393-21ssh00:06:161
281222994220,18cyclictest281743-21fschecks_time19:11:131
28122299421,11cyclictest316629-21ntpq21:06:191
281221994235,4cyclictest439-21systemd-logind00:00:020
281221994233,3cyclictest0-21swapper/021:16:200
281223994132,6cyclictest346283-21proc_pri21:56:222
28122299418,3cyclictest0-21swapper/120:26:071
28122299417,5cyclictest0-21swapper/123:51:271
28122299413,8cyclictest295675-21ntpq19:56:211
281222994134,4cyclictest384494-21cut23:00:011
281222994133,5cyclictest421-21dbus-daemon23:10:021
28122299412,15cyclictest428575-21ntpq00:11:191
28122299412,15cyclictest428575-21ntpq00:11:191
281222994117,21cyclictest0-21swapper/121:26:261
28122299411,6cyclictest300108-21ntpq20:11:171
28122299411,6cyclictest300108-21ntpq20:11:171
28122299411,10cyclictest361487-21ntpq22:21:211
28122299411,10cyclictest283634-21ntpq19:16:191
281221994135,4cyclictest294744-21systemd-run19:55:010
281221994134,5cyclictest421-21dbus-daemon00:30:010
281221994133,5cyclictest340518-21grep21:46:280
281221994133,5cyclictest340518-21grep21:46:280
281221994133,3cyclictest0-21swapper/023:51:210
281223994031,5cyclictest0-21swapper/223:01:202
28122299408,10cyclictest420-21avahi-daemon23:46:081
28122299402,7cyclictest391943-21ssh23:11:231
28122299401,9cyclictest315179-21ntpq21:01:171
28122299401,5cyclictest285116-21ntpq19:21:161
28122299401,11cyclictest443781-21ntpq00:36:211
28122299401,11cyclictest434676-21ntpq00:21:201
28122299401,11cyclictest312688-21sshd20:52:351
28122299401,10cyclictest395016-21ntp_states23:16:241
28122299401,10cyclictest395016-21ntp_states23:16:241
28122299401,10cyclictest309143-21ntpq20:41:171
281222994010,5cyclictest317225-21diskmemload23:00:261
28122199406,19cyclictest358275-21ssh22:16:180
281221994034,4cyclictest694-21cron22:45:010
281221994032,5cyclictest406368-21/usr/sbin/munin23:36:130
281221994028,3cyclictest0-21swapper/021:26:190
281223993933,3cyclictest0-21swapper/222:10:012
281223993933,3cyclictest0-21swapper/222:10:012
281223993919,15cyclictest351ktimers/220:51:192
28122299392,6cyclictest330956-21ntpq21:31:271
28122299392,6cyclictest306209-21ntpq20:31:241
28122299392,5cyclictest401108-21ntpq23:26:241
28122299391,7cyclictest303205-21ntpq20:21:231
281222993914,22cyclictest0-21swapper/121:56:111
28122299391,13cyclictest333823-21ntpq21:36:151
28122299391,10cyclictest297362-21sensors_fan20:01:331
281221993931,5cyclictest1-21systemd19:15:010
281221993931,4cyclictest1003-21mta-sts-daemon23:43:310
281221993929,7cyclictest445496-21cron00:40:010
281223993829,5cyclictest0-21swapper/200:01:212
281223993829,5cyclictest0-21swapper/200:01:212
281223993828,6cyclictest0-21swapper/223:21:202
281223993828,6cyclictest0-21swapper/223:21:202
281223993828,6cyclictest0-21swapper/221:51:242
281223993828,6cyclictest0-21swapper/221:51:242
28122299388,3cyclictest424039-21ssh00:04:421
28122299388,3cyclictest424039-21ssh00:04:421
28122299384,6cyclictest404014-21ntpq23:31:191
28122299383,8cyclictest2763350-21cups-browsed21:51:181
28122299383,8cyclictest2763350-21cups-browsed21:51:181
28122299382,6cyclictest440658-21ntpq00:31:161
28122299382,14cyclictest321588-21ntpq21:16:191
281222993819,16cyclictest1-21systemd22:12:151
281222993819,16cyclictest1-21systemd22:12:151
281222993810,25cyclictest339696-21cat21:46:101
281222993810,25cyclictest339696-21cat21:46:101
28122299380,11cyclictest399478-21ssh23:24:121
28122299380,11cyclictest399478-21ssh23:24:121
28122299380,11cyclictest287216-21sshd19:28:361
281221993828,4cyclictest306794-21latency_hist20:35:020
28122199382,27cyclictest367443-21ssh22:31:180
281223993729,5cyclictest291603-21unixbench_multi19:41:292
281223993728,5cyclictest436369-21munin-plugin-st00:25:012
281223993728,4cyclictest0-21swapper/200:26:102
281223993727,6cyclictest429023-21unixbench_multi00:11:252
281223993727,6cyclictest429023-21unixbench_multi00:11:252
28122299371,9cyclictest358335-21ssh22:16:201
28122299371,8cyclictest290871-21irqstats19:41:121
28122299371,12cyclictest376746-21ntpq22:46:221
28122299371,11cyclictest665-21snmpd21:41:151
28122299371,11cyclictest665-21snmpd21:41:151
28122299370,12cyclictest366594-21ssh22:30:371
281221993731,3cyclictest0-21swapper/023:01:190
281221993730,3cyclictest0-21swapper/022:06:200
281221993730,3cyclictest0-21swapper/022:06:200
281221993729,5cyclictest420-21avahi-daemon23:07:450
281223993628,5cyclictest308268-21latency_hist20:40:022
281223993628,4cyclictest314365-21kworker/u8:10-rpciod21:31:172
28122299361,9cyclictest288075-21ntpq19:31:171
28122299361,5cyclictest370485-21ssh22:36:161
28122299361,13cyclictest318611-21ntpq21:11:231
28122299361,10cyclictest410177-21ntpq23:41:221
28122299361,10cyclictest310573-21ntpq20:46:131
281221993629,3cyclictest1000-21mta-sts-daemon21:50:540
281221993629,3cyclictest1000-21mta-sts-daemon21:50:540
281221993623,10cyclictest394672-21kernelversion23:16:150
281221993623,10cyclictest394672-21kernelversion23:16:150
281223993528,5cyclictest281220-21cyclictest22:30:022
281223993528,4cyclictest375348-21cron22:45:012
281223993528,3cyclictest0-21swapper/223:51:102
281223993528,3cyclictest0-21swapper/222:21:062
281223993526,5cyclictest0-21swapper/223:41:212
28122299351,8cyclictest307592-21/usr/sbin/munin20:36:141
28122299350,9cyclictest294177-21ntp_states19:51:201
28122199354,28cyclictest420-21avahi-daemon21:31:050
281221993531,2cyclictest0-21swapper/022:11:190
281221993531,2cyclictest0-21swapper/022:11:190
281221993529,4cyclictest0-21swapper/023:25:020
281221993529,4cyclictest0-21swapper/023:25:020
281221993528,4cyclictest420-21avahi-daemon22:26:340
281221993527,6cyclictest281220-21cyclictest00:15:000
281221993527,6cyclictest281220-21cyclictest00:15:000
281221993525,7cyclictest303808-21latency_hist20:25:020
28122199352,24cyclictest413163-21ssh23:46:160
281223993426,5cyclictest1001-21mta-sts-daemon21:45:442
281223993426,5cyclictest1001-21mta-sts-daemon21:45:442
281223993418,9cyclictest317225-21diskmemload22:01:202
28122399341,23cyclictest443797-21ssh00:36:212
281221993428,4cyclictest378403-21sh22:50:010
281221993425,7cyclictest421-21dbus-daemon20:45:010
281221993423,3cyclictest0-21swapper/019:50:010
281223993328,2cyclictest0-21swapper/219:20:002
281223993325,5cyclictest293250-21systemd-run19:50:012
281223993324,4cyclictest0-21swapper/221:21:192
281223993323,7cyclictest329542-21munin-plugin-st21:30:012
281223993321,9cyclictest333757-21missed_timers21:36:132
28122399332,16cyclictest318531-21ssh21:11:202
281221993328,3cyclictest0-21swapper/020:01:290
281221993325,6cyclictest424170-21sh00:05:010
281221993325,6cyclictest424170-21sh00:05:010
281221993325,5cyclictest431962-21timerwakeupswit00:16:270
281221993323,7cyclictest315133-21ntp_kernel_pll_21:01:160
281223993225,4cyclictest1003-21mta-sts-daemon22:46:202
281223993224,5cyclictest383307-21unixbench_multi22:56:372
281223993224,5cyclictest314094-21unixbench_multi20:56:292
281223993223,5cyclictest289240-21sort19:36:102
28122399322,21cyclictest413265-21ssh23:46:202
281223993217,7cyclictest0-21swapper/200:06:212
281223993217,7cyclictest0-21swapper/200:06:212
281221993227,2cyclictest0-21swapper/021:40:020
281221993224,5cyclictest296306-21latency_hist20:00:020
281221993222,7cyclictest318850-21switchtime21:11:300
281221993218,9cyclictest349235-21sh22:01:200
281223993124,4cyclictest1001-21mta-sts-daemon00:31:212
281223993123,5cyclictest305154-21unixbench_singl20:26:272
281223993118,10cyclictest0-21swapper/222:11:112
281223993118,10cyclictest0-21swapper/222:11:112
281221993125,3cyclictest338732-21cron21:45:010
281221993125,3cyclictest338732-21cron21:45:010
281221993125,3cyclictest0-21swapper/020:30:020
281221993122,7cyclictest324192-21cpu21:21:080
281221993121,7cyclictest0-21swapper/020:20:000
281221993118,10cyclictest427-21polkitd23:33:160
28122399303,21cyclictest394973-21ssh23:16:222
28122399303,21cyclictest394973-21ssh23:16:222
281223993023,3cyclictest0-21swapper/221:41:152
281223993023,3cyclictest0-21swapper/221:41:152
281223993022,5cyclictest317219-21cron21:10:012
281223993022,5cyclictest1-21systemd22:40:022
281223993019,7cyclictest401426-21timerandwakeup23:26:312
281223993013,13cyclictest1001-21mta-sts-daemon21:16:202
28122299302,5cyclictest421-21dbus-daemon19:10:011
281221993024,4cyclictest290349-21latency_hist19:40:020
281221993018,8cyclictest151ktimers/019:16:210
281223992923,3cyclictest0-21swapper/223:36:342
281223992922,4cyclictest282709-21munin-plugin-st19:15:002
281223992921,5cyclictest298253-21fschecks_count20:06:132
281223992921,5cyclictest285609-21unixbench_multi19:21:272
281221992924,3cyclictest12750irq/33-eth022:36:180
281221992921,3cyclictest0-21swapper/020:51:280
281221992920,6cyclictest2763350-21cups-browsed22:20:200
281221992919,7cyclictest401571-21tail23:26:320
281221992916,10cyclictest313227-21fschecks_count20:56:100
281223992822,3cyclictest0-21swapper/220:45:012
281223992820,5cyclictest2763350-21cups-browsed19:57:502
281223992819,7cyclictest433282-21latency_hist00:20:022
281223992810,14cyclictest351ktimers/220:31:242
281221992822,3cyclictest0-21swapper/020:46:190
281221992822,3cyclictest0-21swapper/020:36:130
281221992820,3cyclictest0-21swapper/000:05:140
281221992820,3cyclictest0-21swapper/000:05:140
281221992818,7cyclictest440116-21tail00:31:070
281221992818,5cyclictest12650irq/32-eth021:56:210
281223992720,3cyclictest0-21swapper/220:11:232
281223992720,3cyclictest0-21swapper/220:11:232
281223992719,5cyclictest2763350-21cups-browsed20:04:382
281223992713,10cyclictest421-21dbus-daemon23:32:102
28122199279,13cyclictest434555-21diskmemload00:21:150
281221992721,4cyclictest290699-21cpuspeed19:41:080
28122399262,18cyclictest391849-21ssh23:11:192
281223992619,4cyclictest301243-21fschecks_count20:16:102
281223992619,4cyclictest287747-21fschecks_count19:31:112
281221992615,8cyclictest265-21systemd-journal19:30:350
28122199261,18cyclictest391897-21ssh23:11:220
281223992516,6cyclictest439-21systemd-logind20:25:012
281221992516,7cyclictest317220-21latency_hist21:10:020
281221992514,8cyclictest0-21swapper/020:11:050
281221992514,8cyclictest0-21swapper/020:11:050
281223992417,4cyclictest1002-21mta-sts-daemon21:05:022
281221992417,4cyclictest282741-21kworker/u8:31-xprtiod19:30:020
28122399232,15cyclictest286620-21ntpq19:26:162
281221992312,9cyclictest151ktimers/019:21:170
28122599223,9cyclictest0-21swapper/322:06:203
28122599223,9cyclictest0-21swapper/322:06:203
28122199229,10cyclictest0-21swapper/019:10:010
28122199221,15cyclictest298632-21ntpq20:06:220
28122599212,9cyclictest0-21swapper/323:31:193
28122599212,10cyclictest0-21swapper/321:41:203
28122599212,10cyclictest0-21swapper/321:41:203
28122599203,7cyclictest0-21swapper/323:11:223
28122599202,9cyclictest0-21swapper/321:31:203
28122599201,10cyclictest0-21swapper/323:21:223
28122599201,10cyclictest0-21swapper/323:21:223
281223992010,7cyclictest281234-21munin-run19:10:012
28122599193,9cyclictest0-21swapper/323:41:223
28122599192,10cyclictest0-21swapper/319:51:193
28122599192,10cyclictest0-21swapper/300:36:173
28122599191,13cyclictest0-21swapper/300:26:233
28122599183,9cyclictest0-21swapper/323:01:193
28122599182,8cyclictest0-21swapper/322:21:093
28122599181,11cyclictest0-21swapper/321:21:193
28122599181,10cyclictest0-21swapper/323:26:213
28122599173,8cyclictest0-21swapper/322:31:213
28122599173,8cyclictest0-21swapper/321:56:143
28122599172,9cyclictest0-21swapper/323:56:223
28122599172,9cyclictest0-21swapper/323:51:243
28122599172,9cyclictest0-21swapper/322:11:283
28122599172,9cyclictest0-21swapper/322:11:283
28122599172,9cyclictest0-21swapper/321:11:193
28122599172,9cyclictest0-21swapper/300:11:043
28122599172,9cyclictest0-21swapper/300:11:043
28122599172,10cyclictest0-21swapper/323:07:203
28122599172,10cyclictest0-21swapper/322:46:213
28122599171,9cyclictest0-21swapper/300:16:213
28122599163,8cyclictest0-21swapper/321:26:183
28122599163,8cyclictest0-21swapper/321:16:193
28122599162,9cyclictest0-21swapper/323:16:253
28122599162,9cyclictest0-21swapper/323:16:253
28122599162,8cyclictest0-21swapper/322:51:203
28122599162,8cyclictest0-21swapper/322:26:203
28122599162,8cyclictest0-21swapper/322:16:183
28122599162,8cyclictest0-21swapper/300:06:163
28122599162,8cyclictest0-21swapper/300:06:163
28122599162,7cyclictest0-21swapper/321:36:063
28122599161,9cyclictest0-21swapper/322:36:203
28122599161,9cyclictest0-21swapper/322:01:163
28122599161,8cyclictest0-21swapper/320:51:203
28122599161,7cyclictest0-21swapper/319:41:163
28122599154,8cyclictest0-21swapper/323:46:193
28122599152,8cyclictest0-21swapper/300:01:193
28122599152,8cyclictest0-21swapper/300:01:193
28122599152,7cyclictest0-21swapper/323:36:213
28122599152,7cyclictest0-21swapper/321:46:163
28122599152,7cyclictest0-21swapper/321:46:163
28122599152,7cyclictest0-21swapper/320:46:203
28122599152,7cyclictest0-21swapper/319:26:183
28122599151,9cyclictest0-21swapper/322:56:093
28122599151,8cyclictest0-21swapper/300:31:113
28122599151,7cyclictest0-21swapper/320:41:183
28122599151,6cyclictest0-21swapper/319:36:183
28122599142,7cyclictest0-21swapper/320:00:013
28122599141,9cyclictest0-21swapper/322:41:063
28122599141,8cyclictest0-21swapper/320:01:243
28122599141,7cyclictest0-21swapper/300:21:083
28122599140,7cyclictest0-21swapper/320:31:173
28122599140,7cyclictest0-21swapper/319:46:203
28122599133,6cyclictest0-21swapper/320:36:133
28122599132,6cyclictest0-21swapper/321:00:013
28122599131,7cyclictest0-21swapper/319:16:113
28122599131,6cyclictest0-21swapper/319:11:193
28122599123,6cyclictest0-21swapper/321:51:203
28122599123,6cyclictest0-21swapper/321:51:203
28122599123,4cyclictest0-21swapper/321:06:173
28122599121,7cyclictest0-21swapper/320:16:213
28122599121,7cyclictest0-21swapper/320:11:093
28122599121,7cyclictest0-21swapper/320:11:093
28122599121,6cyclictest0-21swapper/320:06:133
28122599120,6cyclictest0-21swapper/319:21:153
28122599112,6cyclictest0-21swapper/319:30:353
28122599111,6cyclictest0-21swapper/321:01:153
28122599111,6cyclictest0-21swapper/320:21:153
28122599111,5cyclictest0-21swapper/320:26:093
28122599102,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional