You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 08:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Wed Nov 26, 2025 00:45:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
454021750,3sleep315098-21kworker/3:019:11:103
454021750,3sleep315098-21kworker/3:019:11:103
2574821220,3sleep00-21swapper/000:28:240
2574821220,3sleep00-21swapper/000:28:240
2639721140,3sleep20-21swapper/200:29:462
2639721140,3sleep20-21swapper/200:29:462
1504421140,3chrt0-21swapper/021:46:070
1504421140,3chrt0-21swapper/021:46:070
2909821070,4sleep30-21swapper/323:16:493
2909821070,4sleep30-21swapper/323:16:493
3238421050,3chrt0-21swapper/200:36:172
3238421050,3chrt0-21swapper/200:36:172
266542900,3sleep20-21swapper/221:21:542
134752890,4sleep00-21swapper/019:36:030
134752890,4sleep00-21swapper/019:36:030
138832880,3sleep20-21swapper/221:05:462
138832880,3sleep20-21swapper/221:05:462
15512780,3sleep01392-21/usr/sbin/munin21:31:000
230762760,2chrt0-21swapper/321:55:593
230762760,2chrt0-21swapper/321:55:593
83772710,4sleep30-21swapper/300:07:013
83772710,4sleep30-21swapper/300:07:013
64002690,2sleep06547-21grep19:16:070
282492670,3sleep30-21swapper/300:31:083
282492670,3sleep30-21swapper/300:31:083
66022620,2sleep30-21swapper/323:28:513
66022620,2sleep30-21swapper/323:28:513
206102600,2sleep20-21swapper/219:55:592
206102600,2sleep20-21swapper/219:55:592
4866995726,26cyclictest131rcu_preempt19:40:561
4866995726,26cyclictest131rcu_preempt19:40:561
232022570,2sleep30-21swapper/323:10:583
232022570,2sleep30-21swapper/323:10:583
486799552,40cyclictest29990-21ssh23:55:562
282722550,3sleep00-21swapper/022:39:360
270192550,3sleep20-21swapper/222:00:522
270192550,3sleep20-21swapper/222:00:522
120982550,3chrt0-21swapper/222:57:192
120982550,3chrt0-21swapper/222:57:192
4865995419,24cyclictest38850irq/55-eth021:25:540
4865995419,24cyclictest38850irq/55-eth021:25:540
54012520,2sleep20-21swapper/220:41:032
81632510,3sleep30-21swapper/321:38:183
81632510,3sleep30-21swapper/321:38:183
77612500,2sleep17921-21fschecks_time22:15:591
77612500,2sleep17921-21fschecks_time22:15:591
486699501,9cyclictest131rcu_preempt20:25:561
486699501,9cyclictest131rcu_preempt20:25:561
176552500,2sleep117661-21sshd23:04:261
176552500,2sleep117661-21sshd23:04:261
486699492,39cyclictest9916-21ssh21:40:571
486699492,39cyclictest9916-21ssh21:40:571
38362490,3sleep20-21swapper/220:36:092
38362490,3sleep20-21swapper/220:36:092
486699472,38cyclictest14229-21ssh21:45:571
486699472,38cyclictest14229-21ssh21:45:571
4867994612,17cyclictest27-21ksoftirqd/219:40:562
4867994612,17cyclictest27-21ksoftirqd/219:40:562
486599464,7cyclictest131rcu_preempt23:10:570
486599464,7cyclictest131rcu_preempt23:10:570
23092460,3sleep30-21swapper/300:40:443
23092460,3sleep30-21swapper/300:40:443
147982460,3chrt0-21swapper/200:15:442
147982460,3chrt0-21swapper/200:15:442
486699454,8cyclictest131rcu_preempt21:20:581
4865994533,7cyclictest38850irq/55-eth022:11:000
4865994533,7cyclictest38850irq/55-eth022:11:000
216122450,3sleep10-21swapper/121:54:121
216122450,3sleep10-21swapper/121:54:121
153942450,2sleep00-21swapper/023:39:000
153942450,2sleep00-21swapper/023:39:000
486599445,32cyclictest732-21runrttasks21:15:570
486599445,32cyclictest732-21runrttasks21:15:570
486599442,25cyclictest542-21snmpd21:55:570
486599442,25cyclictest542-21snmpd21:55:570
158962440,4sleep20-21swapper/200:16:042
158962440,4sleep20-21swapper/200:16:042
113232440,3sleep30-21swapper/321:41:203
113232440,3sleep30-21swapper/321:41:203
4866994318,19cyclictest10821-21sh00:10:571
4866994318,19cyclictest10821-21sh00:10:571
186362420,3sleep20-21swapper/223:05:512
186362420,3sleep20-21swapper/223:05:512
486699411,32cyclictest23327-21ssh00:25:571
486699411,32cyclictest23327-21ssh00:25:571
76242390,3sleep20-21swapper/222:15:552
76242390,3sleep20-21swapper/222:15:552
486799392,29cyclictest1260-21ssh22:45:542
486799392,29cyclictest1260-21ssh22:45:542
486799392,29cyclictest1260-21ssh22:45:542
89622380,2sleep08966-21sh00:07:590
89622380,2sleep08966-21sh00:07:590
54252380,3sleep30-21swapper/322:50:453
54252380,3sleep30-21swapper/322:50:453
54252380,3sleep30-21swapper/322:50:453
486799381,33cyclictest26097-21cat20:10:542
486699384,26cyclictest1307-21ssh21:30:521
486699384,26cyclictest1307-21ssh21:30:521
486699383,28cyclictest1394-21ssh21:30:571
94562370,4sleep20-21swapper/223:31:232
94562370,4sleep20-21swapper/223:31:232
486599373,24cyclictest38850irq/55-eth022:15:560
486599373,24cyclictest38850irq/55-eth022:15:560
256582370,2sleep30-21swapper/300:28:153
256582370,2sleep30-21swapper/300:28:153
486799365,24cyclictest11147-21latency_hist19:30:532
486699361,25cyclictest10102-21ssh22:55:551
486699361,25cyclictest10102-21ssh22:55:551
296072360,4sleep10-21swapper/123:55:451
296072360,4sleep10-21swapper/123:55:451
268742360,3sleep30-21swapper/321:22:153
240622360,3chrt0-21swapper/322:34:493
240622360,3chrt0-21swapper/322:34:493
486799353,24cyclictest3348-21ssh22:10:592
486799353,24cyclictest3348-21ssh22:10:592
486599355,3cyclictest0-21swapper/000:15:560
486599355,3cyclictest0-21swapper/000:15:560
486599353,24cyclictest25469-21ssh23:50:560
486599353,24cyclictest25469-21ssh23:50:560
303182350,3sleep10-21swapper/120:21:011
303182350,3sleep10-21swapper/120:21:011
486599344,21cyclictest38850irq/55-eth000:30:550
486599344,21cyclictest38850irq/55-eth000:30:550
60732330,3chrt0-21swapper/221:36:042
60732330,3chrt0-21swapper/221:36:042
4867993317,10cyclictest27448-21sh23:15:582
4867993317,10cyclictest27448-21sh23:15:582
486699333,4cyclictest131rcu_preempt00:01:001
486599334,21cyclictest38850irq/55-eth022:55:560
486599334,21cyclictest38850irq/55-eth022:55:560
486599331,25cyclictest0-21swapper/020:00:560
486599331,25cyclictest0-21swapper/020:00:560
79192320,2sleep20-21swapper/222:52:362
79192320,2sleep20-21swapper/222:52:362
7152320,2sleep00-21swapper/023:59:150
52842320,2sleep215668-21diskmemload23:26:372
52842320,2sleep215668-21diskmemload23:26:372
486799323,21cyclictest14245-21ssh21:45:582
486799323,21cyclictest14245-21ssh21:45:582
486799322,7cyclictest131rcu_preempt19:45:562
486799322,22cyclictest16023-21sh21:10:562
486699324,22cyclictest12371-21ssh23:35:531
486599322,23cyclictest3743-21ssh23:25:570
486599322,23cyclictest3743-21ssh23:25:570
486599322,22cyclictest14489-21sh23:00:580
486599322,22cyclictest14489-21sh23:00:580
486599321,28cyclictest16024-21rm21:10:560
250402320,9sleep012-21ksoftirqd/021:20:590
225622320,2sleep00-21swapper/023:46:240
225622320,2sleep00-21swapper/023:46:240
145382320,3sleep20-21swapper/223:37:382
145382320,3sleep20-21swapper/223:37:382
60772310,3sleep30-21swapper/300:05:363
51982310,2sleep00-21swapper/000:04:120
486699314,22cyclictest24993-21ssh22:35:561
486699311,23cyclictest0-21swapper/122:45:551
486699311,23cyclictest0-21swapper/122:45:551
486699311,23cyclictest0-21swapper/122:45:551
486599312,23cyclictest1282-21ssh22:45:560
486599312,23cyclictest1282-21ssh22:45:560
486599312,23cyclictest1282-21ssh22:45:560
278372310,3sleep20-21swapper/200:31:032
278372310,3sleep20-21swapper/200:31:032
244472310,2sleep20-21swapper/221:20:262
244472310,2sleep20-21swapper/221:20:262
227422310,2sleep122753-21ssh22:32:351
227422310,2sleep122753-21ssh22:32:351
84122300,2sleep28415-21sh00:07:052
84122300,2sleep28415-21sh00:07:052
486799302,20cyclictest18820-21chrt23:05:592
486799302,20cyclictest18820-21chrt23:05:592
486699305,16cyclictest21-21ksoftirqd/120:40:551
486699302,22cyclictest5648-21ssh21:36:011
486699302,22cyclictest5648-21ssh21:36:011
486599302,25cyclictest27450-21ssh23:15:580
486599302,25cyclictest27450-21ssh23:15:580
486599302,21cyclictest20646-21ssh22:30:560
486599302,21cyclictest20646-21ssh22:30:560
486599301,21cyclictest0-21swapper/000:10:560
486599301,21cyclictest0-21swapper/000:10:560
297852300,3sleep20-21swapper/221:26:002
297852300,3sleep20-21swapper/221:26:002
120522300,3sleep30-21swapper/322:20:593
120522300,3sleep30-21swapper/322:20:593
116022300,3sleep00-21swapper/021:41:490
116022300,3sleep00-21swapper/021:41:490
486899291,22cyclictest0-21swapper/300:10:563
486899291,22cyclictest0-21swapper/300:10:563
486799293,20cyclictest16796-21ssh23:40:572
486799292,22cyclictest29831-21taskset20:20:572
486799292,22cyclictest29831-21taskset20:20:572
4866992920,5cyclictest0-21swapper/120:30:561
4866992920,5cyclictest0-21swapper/120:30:561
486699291,22cyclictest31792-21sh23:20:591
486699291,22cyclictest31792-21sh23:20:591
486599292,23cyclictest0-21swapper/020:50:560
486599291,24cyclictest16256-21ssh22:25:560
486599291,24cyclictest16256-21ssh22:25:560
119142290,2sleep00-21swapper/023:35:400
119142290,2sleep00-21swapper/023:35:400
4867992810,14cyclictest21191-21ssh23:45:562
4867992810,14cyclictest21191-21ssh23:45:562
486699287,4cyclictest131rcu_preempt19:15:541
308962280,2sleep130905-21ssh22:41:431
308962280,2sleep130905-21ssh22:41:431
486799273,19cyclictest261rcuc/219:15:542
486599271,22cyclictest18820-21taskset23:05:590
486599271,22cyclictest18820-21taskset23:05:590
486599271,21cyclictest542-21snmpd19:20:560
486599271,21cyclictest542-21snmpd19:20:560
272612270,1sleep027395-21fschecks_time22:00:590
272612270,1sleep027395-21fschecks_time22:00:590
261412270,2sleep20-21swapper/223:51:042
261412270,2sleep20-21swapper/223:51:042
52832260,2chrt0-21swapper/019:15:450
52832260,2chrt0-21swapper/019:15:450
4867992612,9cyclictest4864-21cyclictest22:25:552
4867992612,9cyclictest4864-21cyclictest22:25:552
4866992617,6cyclictest23099-21rm23:10:571
4866992617,6cyclictest23099-21rm23:10:571
486599268,13cyclictest12-21ksoftirqd/021:35:590
486599268,13cyclictest12-21ksoftirqd/021:35:590
486599261,4cyclictest0-21swapper/020:10:590
486599261,4cyclictest0-21swapper/020:10:590
486599261,21cyclictest38850irq/55-eth000:35:570
486599261,21cyclictest38850irq/55-eth000:35:570
486699251,19cyclictest13022-21taskset19:35:591
486699251,19cyclictest13022-21taskset19:35:591
4865992516,6cyclictest0-21swapper/020:15:570
4865992516,6cyclictest0-21swapper/020:15:570
48152255,13sleep20-21swapper/219:11:142
48152255,13sleep20-21swapper/219:11:142
164622250,3chrt0-21swapper/019:45:230
164622250,3chrt0-21swapper/019:45:230
4867992416,4cyclictest27-21ksoftirqd/220:55:572
4867992411,2cyclictest131rcu_preempt00:01:022
4865992412,10cyclictest0-21swapper/020:10:540
486599241,19cyclictest18823-21grep21:51:010
486599241,19cyclictest18823-21grep21:51:010
486599241,17cyclictest31785-21ssh23:20:580
486599241,17cyclictest31785-21ssh23:20:580
191792240,2sleep30-21swapper/300:20:433
191792240,2sleep30-21swapper/300:20:433
486899233,14cyclictest0-21swapper/321:30:523
486899233,14cyclictest0-21swapper/321:30:523
4867992316,4cyclictest27-21ksoftirqd/219:26:102
486699231,18cyclictest11997-21ssh22:20:581
486699231,18cyclictest11997-21ssh22:20:581
486599239,8cyclictest29650irq/49-vc410:36:090
486599233,16cyclictest542-21snmpd20:40:570
4865992317,4cyclictest32022-21ssh22:06:030
4865992317,4cyclictest32022-21ssh22:06:030
4865992317,4cyclictest32022-21ssh22:06:030
486599231,3cyclictest0-21swapper/000:23:350
486599231,3cyclictest0-21swapper/000:23:350
210332230,2sleep20-21swapper/200:21:422
210332230,2sleep20-21swapper/200:21:422
486799227,7cyclictest27-21ksoftirqd/221:41:102
486799227,7cyclictest27-21ksoftirqd/221:41:102
4867992216,4cyclictest27-21ksoftirqd/223:25:112
4867992216,4cyclictest27-21ksoftirqd/223:25:112
486799221,17cyclictest28076-21cut20:16:002
486799221,17cyclictest28076-21cut20:16:002
486799221,16cyclictest24058-21ls23:11:072
486799221,16cyclictest24058-21ls23:11:072
486699223,15cyclictest5719-21ssh22:50:541
486699223,15cyclictest5719-21ssh22:50:541
486699221,19cyclictest27062-21ls20:11:121
486699221,19cyclictest27062-21ls20:11:121
486699221,17cyclictest20603-21sh21:15:561
486699221,17cyclictest20603-21sh21:15:561
486699221,17cyclictest13104-21ssh23:36:041
486699221,17cyclictest13104-21ssh23:36:041
486599226,12cyclictest12-21ksoftirqd/020:20:560
486599226,12cyclictest12-21ksoftirqd/020:20:560
486599222,16cyclictest10331-21apt-get20:55:550
486599222,14cyclictest17103-21fw_packets23:41:000
4865992214,6cyclictest12641-21/usr/sbin/munin22:21:050
4865992214,6cyclictest12641-21/usr/sbin/munin22:21:050
36062222,15sleep13618-21apt-get19:10:571
36062222,15sleep13618-21apt-get19:10:571
110522220,8sleep121-21ksoftirqd/123:34:111
486799218,10cyclictest27931-21ssh22:39:012
4867992114,5cyclictest19192-21cat19:51:022
486799211,17cyclictest23957-21ls20:05:442
486799211,17cyclictest23957-21ls20:05:442
486799211,15cyclictest15685-21ls21:10:432
486799211,15cyclictest15685-21ls21:10:432
4867992111,5cyclictest0-21swapper/219:30:572
4867992111,5cyclictest0-21swapper/219:30:572
4866992114,5cyclictest5602-21ssh23:27:091
4866992114,5cyclictest5602-21ssh23:27:091
4866992113,5cyclictest0-21swapper/121:00:441
4866992113,5cyclictest0-21swapper/100:20:561
4866992113,5cyclictest0-21swapper/100:20:561
486699211,17cyclictest6521-21ssh00:05:571
486699211,17cyclictest6521-21ssh00:05:571
486699211,17cyclictest13802-21ls21:05:431
486699211,17cyclictest13802-21ls21:05:431
486699211,16cyclictest28490-21ls23:16:091
486699211,16cyclictest28490-21ls23:16:091
486699210,14cyclictest22889-21ls23:10:491
486699210,14cyclictest22889-21ls23:10:491
110352210,10sleep333-21ksoftirqd/322:56:073
110352210,10sleep333-21ksoftirqd/322:56:073
52662200,3sleep20-21swapper/221:35:402
4867992014,4cyclictest27-21ksoftirqd/220:11:042
4867992014,4cyclictest27-21ksoftirqd/220:11:042
4867992013,4cyclictest27-21ksoftirqd/220:31:352
4867992013,4cyclictest27-21ksoftirqd/220:31:352
486699201,15cyclictest15250-21sh00:15:571
486699201,15cyclictest15250-21sh00:15:571
486699201,14cyclictest25785-21apt-get20:10:541
486599207,11cyclictest11107-21latency_hist19:30:510
486599207,11cyclictest11107-21latency_hist19:30:510
486599201,9cyclictest15802-21latency_hist21:10:480
486599201,9cyclictest15802-21latency_hist21:10:480
486599201,10cyclictest6308-21needreboot22:51:000
486599201,10cyclictest6308-21needreboot22:51:000
486599201,10cyclictest30386-21timerwakeupswit22:41:110
486599201,10cyclictest30386-21timerwakeupswit22:41:110
486899192,11cyclictest0-21swapper/320:30:583
486899192,11cyclictest0-21swapper/320:30:583
486899191,12cyclictest0-21swapper/319:30:513
486899191,12cyclictest0-21swapper/319:30:513
486899191,11cyclictest0-21swapper/321:51:023
486899191,11cyclictest0-21swapper/321:51:023
486799198,3cyclictest27-21ksoftirqd/222:07:272
486799198,3cyclictest27-21ksoftirqd/222:07:272
486799198,3cyclictest27-21ksoftirqd/222:07:272
4867991914,3cyclictest27-21ksoftirqd/220:26:102
4867991914,3cyclictest27-21ksoftirqd/220:26:102
486799190,16cyclictest20783-21grep22:30:582
486799190,16cyclictest20783-21grep22:30:582
486699199,8cyclictest20584-21df_inode19:55:571
486699199,8cyclictest20584-21df_inode19:55:571
486699191,4cyclictest131rcu_preempt23:55:591
4866991914,3cyclictest32137-21timerandwakeup00:36:161
4866991914,3cyclictest32137-21timerandwakeup00:36:161
486699191,2cyclictest0-21swapper/120:01:091
486699191,2cyclictest0-21swapper/120:01:091
4866991911,6cyclictest4536-21cron20:40:441
4866991911,6cyclictest4536-21cron20:40:441
486699191,13cyclictest17017-21fschecks_time23:41:001
486699190,17cyclictest23286-21ls21:56:011
486699190,17cyclictest23286-21ls21:56:011
486599192,3cyclictest131rcu_preempt19:55:580
486599192,3cyclictest131rcu_preempt19:55:580
4865991914,3cyclictest12-21ksoftirqd/021:01:000
4865991914,3cyclictest12-21ksoftirqd/021:01:000
4865991913,4cyclictest12-21ksoftirqd/020:26:090
4865991913,4cyclictest12-21ksoftirqd/020:26:090
486899184,10cyclictest0-21swapper/322:35:563
486899182,11cyclictest0-21swapper/320:45:583
486899181,13cyclictest0-21swapper/323:10:443
486899181,13cyclictest0-21swapper/323:10:443
486899181,13cyclictest0-21swapper/322:01:093
486899181,13cyclictest0-21swapper/322:01:093
486899181,11cyclictest0-21swapper/323:25:513
486899181,11cyclictest0-21swapper/323:25:513
486899181,11cyclictest0-21swapper/320:55:573
486799184,12cyclictest27-21ksoftirqd/222:40:572
486799184,12cyclictest27-21ksoftirqd/222:40:572
4867991813,3cyclictest27-21ksoftirqd/219:25:442
4867991813,3cyclictest27-21ksoftirqd/219:25:442
4867991812,4cyclictest27-21ksoftirqd/220:55:112
4867991812,4cyclictest27-21ksoftirqd/220:45:592
486799181,11cyclictest13308-21ls19:36:012
486799181,11cyclictest13308-21ls19:36:012
4867991810,6cyclictest28571-21sshd22:01:252
4867991810,6cyclictest28571-21sshd22:01:252
4867991810,6cyclictest19119-21ssh21:51:032
4867991810,6cyclictest19119-21ssh21:51:032
486699183,11cyclictest0-21swapper/120:45:581
486699183,10cyclictest0-21swapper/119:30:511
486699183,10cyclictest0-21swapper/119:30:511
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional