You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 07:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #4, slot #1

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1s.osadl.org (updated Sun Apr 19, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
727221370,3chrt0-21swapper/200:17:172
727221370,3chrt0-21swapper/200:17:172
727221370,3chrt0-21swapper/200:17:172
1234021240,1sleep00-21swapper/000:32:140
233521070,3sleep20-21swapper/200:02:412
233521070,3sleep20-21swapper/200:02:412
67152870,3sleep05973-21/usr/sbin/munin21:22:200
67152870,3sleep05973-21/usr/sbin/munin21:22:200
67152870,3sleep05973-21/usr/sbin/munin21:22:200
310322830,3chrt0-21swapper/221:02:122
310322830,3chrt0-21swapper/221:02:122
310322830,3chrt0-21swapper/221:02:122
260552800,3sleep20-21swapper/219:22:142
172772670,3chrt0-21swapper/223:17:192
172772670,3chrt0-21swapper/223:17:192
126222670,3sleep20-21swapper/200:32:162
126222670,3sleep20-21swapper/200:32:162
216432650,2chrt21640-21ntpq22:02:211
216432650,2chrt21640-21ntpq22:02:211
216432650,2chrt21640-21ntpq22:02:211
8772630,3sleep20-21swapper/200:02:052
8772630,3sleep20-21swapper/200:02:052
16692630,2sleep10-21swapper/119:42:191
16692630,2sleep10-21swapper/119:42:191
103722630,2chrt0-21swapper/221:32:192
103722630,2chrt0-21swapper/221:32:192
272162610,1sleep0201rcuc/020:52:130
215752520,1sleep1271rcuc/120:37:141
215752520,1sleep1271rcuc/120:37:141
215752520,1sleep1271rcuc/120:37:141
303852490,4chrt0-21swapper/223:52:202
303852490,4chrt0-21swapper/223:52:202
20402470,2chrt1984-21cron21:12:041
20402470,2chrt1984-21cron21:12:041
20402470,2chrt1984-21cron21:12:041
184522460,2sleep20-21swapper/220:27:182
184522460,2sleep20-21swapper/220:27:182
184522460,2sleep20-21swapper/220:27:182
172342420,3sleep20-21swapper/221:52:132
172342420,3sleep20-21swapper/221:52:132
172342420,3sleep20-21swapper/221:52:132
2227099402,20cyclictest15162irqstats00:02:171
2227099402,20cyclictest15162irqstats00:02:171
22269994034,4cyclictest674-21gpm21:42:240
22269994034,4cyclictest674-21gpm21:42:240
124382400,2chrt142-21systemd-journal23:04:510
124382400,2chrt142-21systemd-journal23:04:510
257912360,3sleep20-21swapper/222:12:232
257912360,3sleep20-21swapper/222:12:232
22270993628,6cyclictest550-21cron22:57:021
22270993628,6cyclictest550-21cron22:57:021
22269993632,2cyclictest0-21swapper/000:02:080
22269993632,2cyclictest0-21swapper/000:02:080
22269993630,4cyclictest572-21cups-browsed23:42:220
22269993630,4cyclictest572-21cups-browsed23:42:220
191392360,2sleep019138-21df21:57:130
191392360,2sleep019138-21df21:57:130
22270993529,4cyclictest22261-21latency_hist23:32:131
22270993529,4cyclictest22261-21latency_hist23:32:131
22270993529,4cyclictest22261-21latency_hist23:32:131
22269993527,6cyclictest14717-21netstat20:17:180
22269993527,6cyclictest14717-21netstat20:17:180
82182340,4sleep07802-21/usr/sbin/munin21:27:140
82182340,4sleep07802-21/usr/sbin/munin21:27:140
82182340,4sleep07802-21/usr/sbin/munin21:27:140
165462330,2sleep20-21swapper/220:22:162
165462330,2sleep20-21swapper/220:22:162
63852320,2sleep20-21swapper/219:57:032
63852320,2sleep20-21swapper/219:57:032
22270993224,6cyclictest30595-21threads19:32:241
22269993227,4cyclictest572-21cups-browsed21:02:210
22269993227,4cyclictest572-21cups-browsed21:02:210
22269993227,3cyclictest0-21swapper/000:02:150
22269993227,3cyclictest0-21swapper/000:02:150
22270993125,4cyclictest7155-21unixbench_multi21:22:241
22270993125,4cyclictest7155-21unixbench_multi21:22:241
22270993125,4cyclictest7155-21unixbench_multi21:22:241
22270993123,6cyclictest8749-21fschecks_time00:22:151
22270993123,6cyclictest8749-21fschecks_time00:22:151
22269993126,3cyclictest22257-21cyclictest20:45:550
22269993126,3cyclictest22257-21cyclictest20:45:550
22269993124,6cyclictest11327-21sshd21:36:540
22269993124,6cyclictest11327-21sshd21:36:540
147252310,3sleep30-21swapper/320:17:193
147252310,3sleep30-21swapper/320:17:193
2227299301,21cyclictest891-21taskset00:02:063
2227299301,21cyclictest891-21taskset00:02:063
22270993024,5cyclictest3273-21latency_hist22:42:011
22270993024,5cyclictest3273-21latency_hist22:42:011
22270993024,5cyclictest3273-21latency_hist22:42:011
22270993023,5cyclictest23883-21kernelversion20:42:161
22270993023,5cyclictest23883-21kernelversion20:42:161
22270993021,7cyclictest15441-21fschecks_count21:47:141
22270993021,6cyclictest30327-21needreboot23:52:191
22270993021,6cyclictest30327-21needreboot23:52:191
22269993026,3cyclictest8209-21latency_hist20:02:010
22269993026,3cyclictest8209-21latency_hist20:02:010
22269993026,2cyclictest0-21swapper/021:22:010
22269993026,2cyclictest0-21swapper/021:22:010
22269993025,4cyclictest572-21cups-browsed20:02:240
22269993025,4cyclictest572-21cups-browsed20:02:240
22269993023,5cyclictest31418-21latency_hist19:37:090
22269993023,5cyclictest2328-21netstat22:37:200
22269993023,5cyclictest2328-21netstat22:37:200
22269993023,5cyclictest2328-21netstat22:37:200
22269993023,5cyclictest11981-21latency_hist20:12:030
22270992924,3cyclictest23746-21switchtime22:07:231
22270992924,3cyclictest23746-21switchtime22:07:231
22270992924,3cyclictest23746-21switchtime22:07:231
22270992923,4cyclictest31916-21grep22:32:011
22270992923,4cyclictest31916-21grep22:32:011
22270992923,3cyclictest0-21swapper/121:42:021
22270992923,3cyclictest0-21swapper/121:42:021
22270992923,3cyclictest0-21swapper/121:42:021
22270992921,6cyclictest4900-21processes21:17:221
22270992921,6cyclictest4900-21processes21:17:221
22269992924,3cyclictest22575-21cat22:07:010
22269992924,3cyclictest22575-21cat22:07:010
22269992924,3cyclictest22575-21cat22:07:010
22269992924,3cyclictest22257-21cyclictest20:57:450
22269992924,3cyclictest22257-21cyclictest20:57:450
22269992924,3cyclictest22257-21cyclictest20:57:450
22269992924,3cyclictest0-21swapper/023:22:240
22269992924,3cyclictest0-21swapper/023:22:240
22269992924,3cyclictest0-21swapper/023:22:240
22269992923,4cyclictest10734-21cut00:27:180
22269992922,5cyclictest428-21sshd19:39:350
22269992922,5cyclictest428-21sshd19:39:350
22269992922,5cyclictest29431-21latency_hist23:52:010
22269992922,5cyclictest29431-21latency_hist23:52:010
22269992922,5cyclictest29431-21latency_hist23:52:010
166182290,3chrt0-21swapper/021:47:480
166182290,3chrt0-21swapper/021:47:480
166182290,3chrt0-21swapper/021:47:480
282122280,3sleep20-21swapper/219:27:172
282122280,3sleep20-21swapper/219:27:172
22270992824,2cyclictest0-21swapper/121:32:011
22270992824,2cyclictest0-21swapper/121:32:011
22270992824,2cyclictest0-21swapper/121:32:011
22270992823,4cyclictest1346-21avahi-daemon22:17:231
22270992823,4cyclictest1346-21avahi-daemon22:17:231
22270992823,4cyclictest1346-21avahi-daemon22:17:231
22270992823,3cyclictest17938-21kworker/u9:0-xprtiod00:02:111
22270992823,3cyclictest17938-21kworker/u9:0-xprtiod00:02:111
22270992821,5cyclictest7506-21switchtime19:57:221
22270992821,5cyclictest7506-21switchtime19:57:221
22270992821,5cyclictest31526-21needreboot21:02:171
22270992821,5cyclictest31526-21needreboot21:02:171
22270992821,5cyclictest30602-21cron21:02:011
22270992821,5cyclictest30602-21cron21:02:011
22270992821,5cyclictest30602-21cron21:02:011
22269992822,4cyclictest572-21cups-browsed22:22:220
22269992822,4cyclictest572-21cups-browsed22:22:220
22269992821,5cyclictest1346-21avahi-daemon21:37:230
22269992821,5cyclictest1346-21avahi-daemon21:37:230
22269992821,5cyclictest1346-21avahi-daemon21:37:230
22270992722,3cyclictest8793-21gltestperf20:02:171
22270992722,3cyclictest8793-21gltestperf20:02:171
22270992722,3cyclictest15062-21gltestperf23:12:151
22270992722,3cyclictest15062-21gltestperf23:12:151
22270992721,5cyclictest6386-21sshd00:16:551
22270992721,5cyclictest6386-21sshd00:16:551
22270992721,5cyclictest6386-21sshd00:16:551
22270992719,6cyclictest19814-21fschecks_time20:32:121
22270992719,6cyclictest19814-21fschecks_time20:32:121
22270992718,7cyclictest10869-21kernelversion00:27:191
22270992718,7cyclictest10869-21kernelversion00:27:191
22269992723,2cyclictest0-21swapper/022:42:130
22269992723,2cyclictest0-21swapper/022:42:130
22269992722,4cyclictest22257-21cyclictest19:42:150
22269992722,4cyclictest22257-21cyclictest19:42:150
22269992721,5cyclictest8272-21sendmail00:22:020
22269992721,5cyclictest8272-21sendmail00:22:020
22269992721,5cyclictest8272-21sendmail00:22:020
22269992721,5cyclictest23289-21kernelversion22:07:170
22269992721,5cyclictest23289-21kernelversion22:07:170
22269992721,5cyclictest23289-21kernelversion22:07:170
22269992720,5cyclictest22515-21threads20:37:250
22269992720,5cyclictest22515-21threads20:37:250
22269992720,5cyclictest22515-21threads20:37:250
66232260,2sleep20-21swapper/221:22:182
66232260,2sleep20-21swapper/221:22:182
66232260,2sleep20-21swapper/221:22:182
22270992621,4cyclictest30053-21latency_hist22:27:021
22270992621,4cyclictest30053-21latency_hist22:27:021
22270992621,4cyclictest20441-21grep23:27:141
22270992621,4cyclictest20441-21grep23:27:141
22270992620,5cyclictest16119-21processes21:47:221
22270992620,5cyclictest16119-21processes21:47:221
22270992620,5cyclictest16119-21processes21:47:221
22270992620,4cyclictest4537-21latency_hist00:12:131
22270992620,4cyclictest4537-21latency_hist00:12:131
22270992620,4cyclictest16847-21gltestperf23:17:141
22270992620,4cyclictest16847-21gltestperf23:17:141
22270992620,4cyclictest14679-21head21:42:241
22270992620,4cyclictest142-21systemd-journal20:22:011
22270992620,4cyclictest142-21systemd-journal20:22:011
22270992619,5cyclictest3952-21latency_hist21:17:051
22270992619,5cyclictest3952-21latency_hist21:17:051
22270992616,8cyclictest24760-21tr23:37:191
22270992616,8cyclictest24760-21tr23:37:191
22270992616,8cyclictest24760-21tr23:37:191
22270992616,8cyclictest142-21systemd-journal19:27:021
2226999269,15cyclictest20341-21munin-node22:19:110
2226999269,15cyclictest20341-21munin-node22:19:110
2226999269,15cyclictest20341-21munin-node22:19:110
22269992622,3cyclictest22257-21cyclictest00:23:270
22269992622,3cyclictest22257-21cyclictest00:23:270
22269992622,2cyclictest0-21swapper/022:47:200
22269992622,2cyclictest0-21swapper/022:47:200
22269992622,2cyclictest0-21swapper/020:47:150
22269992621,3cyclictest0-21swapper/020:32:160
22269992621,3cyclictest0-21swapper/020:32:160
22269992620,5cyclictest30293-21missed_timers23:52:180
22269992620,5cyclictest30293-21missed_timers23:52:180
22269992620,4cyclictest22257-21cyclictest22:36:380
22269992620,4cyclictest22257-21cyclictest22:36:380
22269992619,4cyclictest0-21swapper/020:27:140
115292260,2sleep111532-21latency23:02:181
115292260,2sleep111532-21latency23:02:181
22270992521,3cyclictest23819-21cron23:37:011
22270992521,3cyclictest23819-21cron23:37:011
22270992521,2cyclictest0-21swapper/123:47:111
22270992521,2cyclictest0-21swapper/123:47:111
22270992521,2cyclictest0-21swapper/121:32:141
22270992521,2cyclictest0-21swapper/121:32:141
22270992520,3cyclictest0-21swapper/123:52:011
22270992520,3cyclictest0-21swapper/123:52:011
22270992520,3cyclictest0-21swapper/123:52:011
22270992520,3cyclictest0-21swapper/122:12:161
22270992520,3cyclictest0-21swapper/122:12:161
22270992520,3cyclictest0-21swapper/119:47:141
22270992520,3cyclictest0-21swapper/119:47:141
22270992519,5cyclictest26884-21latency_hist20:52:011
22270992519,4cyclictest32599-21unixbench-2d19:37:241
22270992519,4cyclictest32599-21unixbench-2d19:37:241
22270992519,4cyclictest20194-21unixbench-2d21:57:251
22270992519,4cyclictest20194-21unixbench-2d21:57:251
22270992519,4cyclictest142-21systemd-journal20:27:151
22270992519,4cyclictest11558-21users20:07:261
22270992519,4cyclictest0-21swapper/119:27:261
22270992519,4cyclictest0-21swapper/119:27:261
22270992517,6cyclictest4822-21sshd22:42:551
22270992517,6cyclictest4822-21sshd22:42:551
22270992514,9cyclictest142-21systemd-journal22:32:131
22270992514,9cyclictest142-21systemd-journal22:32:131
22269992521,2cyclictest0-21swapper/023:12:240
22269992521,2cyclictest0-21swapper/023:12:240
22269992521,2cyclictest0-21swapper/021:12:150
22269992521,2cyclictest0-21swapper/021:12:150
22269992521,2cyclictest0-21swapper/020:27:220
22269992521,2cyclictest0-21swapper/020:27:220
22269992521,2cyclictest0-21swapper/020:27:220
22269992520,3cyclictest0-21swapper/021:07:170
22269992520,3cyclictest0-21swapper/021:07:170
22269992520,3cyclictest0-21swapper/021:07:170
22269992520,3cyclictest0-21swapper/019:52:220
22269992520,3cyclictest0-21swapper/019:52:220
22269992519,4cyclictest0-21swapper/019:27:020
22269992518,5cyclictest22837-21needreboot23:32:190
22269992518,5cyclictest22837-21needreboot23:32:190
22269992517,6cyclictest13762-21sshd00:35:370
22269992517,6cyclictest13762-21sshd00:35:370
66352240,1sleep20-21swapper/219:57:122
66352240,1sleep20-21swapper/219:57:122
324112240,3sleep30-21swapper/322:32:133
324112240,3sleep30-21swapper/322:32:133
22270992420,2cyclictest0-21swapper/120:22:201
22270992420,2cyclictest0-21swapper/120:22:201
22270992419,3cyclictest28218-21timerwakeupswit20:52:241
22270992419,3cyclictest21271-21latency_hist20:37:101
22270992419,3cyclictest21271-21latency_hist20:37:101
22270992418,4cyclictest6989-21gltestperf00:17:141
22270992418,4cyclictest6989-21gltestperf00:17:141
22270992418,4cyclictest6989-21gltestperf00:17:141
22270992418,4cyclictest18039-21sendmail_mailqu21:52:221
22270992418,4cyclictest18039-21sendmail_mailqu21:52:221
22269992420,2cyclictest0-21swapper/021:52:190
22269992420,2cyclictest0-21swapper/021:52:190
22269992420,2cyclictest0-21swapper/000:12:220
22269992420,2cyclictest0-21swapper/000:12:220
22269992420,2cyclictest0-21swapper/000:12:220
22269992419,3cyclictest0-21swapper/022:15:040
22269992419,3cyclictest0-21swapper/022:15:040
22269992418,5cyclictest13690-21rt-features23:07:220
22269992418,5cyclictest13690-21rt-features23:07:220
22269992418,4cyclictest0-21swapper/023:37:220
22269992418,4cyclictest0-21swapper/023:37:220
22269992418,4cyclictest0-21swapper/023:37:220
22269992418,4cyclictest0-21swapper/022:52:210
22269992418,4cyclictest0-21swapper/022:52:210
22269992418,4cyclictest0-21swapper/019:47:190
22269992418,4cyclictest0-21swapper/019:47:190
22269992416,6cyclictest9946-21rt-features22:57:230
22269992416,6cyclictest9946-21rt-features22:57:230
22269992410,12cyclictest0-21swapper/020:24:250
22270992319,2cyclictest0-21swapper/122:52:141
22270992319,2cyclictest0-21swapper/122:52:141
22270992318,3cyclictest19504-21timerandwakeup23:22:251
22270992318,3cyclictest0-21swapper/120:17:011
22270992316,3cyclictest0-21swapper/123:12:131
22269992318,2cyclictest0-21swapper/020:12:240
22269992315,6cyclictest22418-21cut23:32:140
22269992315,6cyclictest22418-21cut23:32:140
22269992310,11cyclictest0-21swapper/000:09:040
22269992310,11cyclictest0-21swapper/000:09:040
22270992217,3cyclictest0-21swapper/123:02:011
22270992217,3cyclictest0-21swapper/123:02:011
22270992216,4cyclictest0-21swapper/100:37:161
22270992216,4cyclictest0-21swapper/100:37:161
22270992216,4cyclictest0-21swapper/100:37:161
22269992217,3cyclictest0-21swapper/000:37:140
22269992217,3cyclictest0-21swapper/000:37:140
22269992217,3cyclictest0-21swapper/000:37:140
22269992215,5cyclictest617-21lldpd22:27:200
22269992215,5cyclictest617-21lldpd22:27:200
22269992215,5cyclictest21026-21ntp_kernel_err23:27:200
46262210,2sleep20-21swapper/219:52:092
46262210,2sleep20-21swapper/219:52:092
280852210,3sleep20-21swapper/220:52:232
22270992117,3cyclictest12634-21idleruntime00:32:161
22270992117,3cyclictest12634-21idleruntime00:32:161
22270992117,2cyclictest0-21swapper/119:52:241
22270992117,2cyclictest0-21swapper/119:52:241
22270992115,4cyclictest0-21swapper/123:12:121
22269992116,3cyclictest0-21swapper/023:19:020
22269992116,3cyclictest0-21swapper/023:19:020
22269992116,3cyclictest0-21swapper/019:27:170
22269992116,3cyclictest0-21swapper/019:27:170
2227299205,9cyclictest0-21swapper/300:02:173
2227299205,9cyclictest0-21swapper/300:02:173
45132190,2chrt0-21swapper/221:17:152
45132190,2chrt0-21swapper/221:17:152
22271991914,3cyclictest277262sleep223:47:112
22271991914,3cyclictest277262sleep223:47:112
2227299189,6cyclictest301512sleep323:52:173
2227299189,6cyclictest301512sleep323:52:173
2227299183,9cyclictest0-21swapper/319:55:533
2227299183,9cyclictest0-21swapper/319:55:533
2227299181,13cyclictest4739-21chrt22:42:223
2227299181,13cyclictest4739-21chrt22:42:223
2227299181,12cyclictest30584-21chrt22:27:153
2227299181,12cyclictest30584-21chrt22:27:153
206962180,3chrt0-21swapper/223:27:162
206962180,3chrt0-21swapper/223:27:162
206962180,3chrt0-21swapper/223:27:162
165762180,3sleep30-21swapper/323:17:123
165762180,3sleep30-21swapper/323:17:123
2227299173,9cyclictest0-21swapper/322:08:533
2227299173,9cyclictest0-21swapper/322:08:533
2227299173,9cyclictest0-21swapper/322:08:533
2227299173,9cyclictest0-21swapper/321:18:543
2227299173,9cyclictest0-21swapper/321:18:543
2227299172,11cyclictest0-21swapper/300:22:173
2227299172,11cyclictest0-21swapper/300:22:173
22272991711,4cyclictest327652sleep319:37:563
22272991711,4cyclictest327652sleep319:37:563
2227299171,12cyclictest20781-21taskset20:32:223
2227299171,12cyclictest20781-21taskset20:32:223
2227299171,12cyclictest13951-21chrt00:37:033
2227299171,12cyclictest13951-21chrt00:37:033
2227299171,11cyclictest7717-21chrt22:52:153
2227299171,11cyclictest7717-21chrt22:52:153
2227299171,11cyclictest15261-21taskset00:37:243
2227299171,11cyclictest15261-21taskset00:37:243
2227299171,11cyclictest15261-21taskset00:37:243
2227299171,11cyclictest0-21swapper/322:02:253
2227299171,11cyclictest0-21swapper/322:02:253
2227299171,11cyclictest0-21swapper/322:02:253
2227199176,9cyclictest20298-21taskset20:32:172
2227199176,9cyclictest20298-21taskset20:32:172
2227199171,13cyclictest14260-21chrt20:17:132
2227199171,13cyclictest14260-21chrt20:17:132
35672161,3sleep30-21swapper/321:13:113
35672161,3sleep30-21swapper/321:13:113
2227299163,9cyclictest0-21swapper/321:42:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional