You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:08
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Sat Jun 28, 2025 09:50:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
177927630,12sleep012-21ksoftirqd/012:40:290
177927630,12sleep012-21ksoftirqd/012:40:290
2503526900,10sleep30-21swapper/321:52:413
2503526900,10sleep30-21swapper/321:52:413
413326300,16sleep2131rcu_preempt00:35:512
181725920,7sleep20-21swapper/209:38:192
181725920,7sleep20-21swapper/209:38:192
106925200,7sleep20-21swapper/209:38:282
106925200,7sleep20-21swapper/209:38:282
2260824680,10sleep10-21swapper/100:25:341
2260824680,10sleep10-21swapper/100:25:341
3138824600,5sleep10-21swapper/100:50:441
1521324560,11sleep10-21swapper/121:35:331
1521324560,11sleep10-21swapper/121:35:331
2347024220,10sleep00-21swapper/000:30:510
2347024220,10sleep00-21swapper/000:30:510
1818324150,7sleep30-21swapper/321:37:303
1818324150,7sleep30-21swapper/321:37:303
1478724150,7sleep20-21swapper/209:45:192
1478724150,7sleep20-21swapper/209:45:192
1478724150,7sleep20-21swapper/209:45:192
1177924040,12sleep30-21swapper/312:22:223
1388224000,4sleep3321rcuc/311:00:523
1621523960,5sleep2261rcuc/201:35:542
1621523960,5sleep2261rcuc/201:35:542
932623880,8sleep10-21swapper/109:40:341
932623880,8sleep10-21swapper/109:40:341
1582523820,11sleep30-21swapper/309:46:063
1582523820,11sleep30-21swapper/309:46:063
2620123800,7sleep30-21swapper/321:46:273
2620123800,7sleep30-21swapper/321:46:273
365323790,6sleep20-21swapper/222:20:532
499623780,18sleep333-21ksoftirqd/309:33:493
499623780,18sleep333-21ksoftirqd/309:33:493
243723770,10sleep20-21swapper/209:31:002
243723770,10sleep20-21swapper/209:31:002
1497623750,6sleep10-21swapper/111:05:381
2405423730,3sleep00-21swapper/010:00:470
186623670,15sleep333-21ksoftirqd/321:44:543
186623670,15sleep333-21ksoftirqd/321:44:543
465423650,5sleep30-21swapper/303:40:363
465423650,5sleep30-21swapper/303:40:363
2633023630,3sleep10-21swapper/109:29:021
2633023630,3sleep10-21swapper/109:29:021
2524523600,3sleep30-21swapper/313:20:493
3018323590,5sleep30-21swapper/321:31:443
3018323590,5sleep30-21swapper/321:31:443
2703623590,9sleep10-21swapper/121:47:181
2703623590,9sleep10-21swapper/121:47:181
781523500,5sleep10-21swapper/121:51:211
1350823490,14sleep1211ksoftirqd/111:05:531
1350823490,14sleep1211ksoftirqd/111:05:531
541923430,7sleep20-21swapper/213:35:512
541923430,7sleep20-21swapper/213:35:512
452523380,6sleep30-21swapper/322:10:323
452523380,6sleep30-21swapper/322:10:323
1892223380,7sleep30-21swapper/312:40:573
2890523350,9sleep10-21swapper/122:00:501
2890523350,9sleep10-21swapper/122:00:501
1330423340,5sleep30-21swapper/309:44:013
1330423340,5sleep30-21swapper/309:44:013
2944223300,5sleep20-21swapper/210:11:012
2944223300,5sleep20-21swapper/210:11:012
2742623280,7sleep30-21swapper/321:50:223
103523270,4sleep20-21swapper/209:30:402
103523270,4sleep20-21swapper/209:30:402
493323240,4sleep10-21swapper/122:10:341
493323240,4sleep10-21swapper/122:10:341
1492023200,8sleep10-21swapper/101:35:451
1492023200,8sleep10-21swapper/101:35:451
1492023200,8sleep10-21swapper/101:35:451
2415123180,3sleep00-21swapper/009:29:100
2415123180,3sleep00-21swapper/009:29:100
2859823150,4sleep30-21swapper/310:15:373
2859823150,4sleep30-21swapper/310:15:373
2612523130,6sleep026131-21latencyplot09:29:030
2612523130,6sleep026131-21latencyplot09:29:030
1348223100,13sleep333-21ksoftirqd/303:45:503
1348223100,13sleep333-21ksoftirqd/303:45:503
1620323080,15sleep3131rcu_preempt21:34:353
1620323080,15sleep3131rcu_preempt21:34:353
1625723060,4sleep20-21swapper/221:29:092
1625723060,4sleep20-21swapper/221:29:092
2639723000,6sleep20-21swapper/221:48:162
2758922990,6sleep20-21swapper/209:29:072
2758922990,6sleep20-21swapper/209:29:072
2433122980,6sleep00-21swapper/021:45:030
2433122980,6sleep00-21swapper/021:45:030
1278622980,6sleep10-21swapper/109:44:411
2550622890,8sleep10-21swapper/100:30:591
2550622890,8sleep10-21swapper/100:30:591
2550622890,8sleep10-21swapper/100:30:591
206822890,7sleep30-21swapper/309:50:213
206822890,7sleep30-21swapper/309:50:213
2831322880,3sleep30-21swapper/300:40:503
2831322880,3sleep30-21swapper/300:40:503
813122870,6sleep10-21swapper/122:15:581
813122870,6sleep10-21swapper/122:15:581
1797822870,2sleep30-21swapper/309:50:443
1797822870,2sleep30-21swapper/309:50:443
1654722850,8sleep30-21swapper/303:50:443
705422810,3sleep20-21swapper/212:30:432
705422810,3sleep20-21swapper/212:30:432
1544522810,5chrt0-21swapper/301:35:513
1544522810,5chrt0-21swapper/301:35:513
1544522810,5chrt0-21swapper/301:35:513
1286022770,7sleep30-21swapper/321:31:363
1286022770,7sleep30-21swapper/321:31:363
1216322770,8sleep00-21swapper/012:30:390
1216322770,8sleep00-21swapper/012:30:390
1308422710,4sleep10-21swapper/101:26:041
2083822690,3sleep30-21swapper/309:29:013
2083822690,3sleep30-21swapper/309:29:013
3240722680,4sleep00-21swapper/013:20:490
3240722680,4sleep00-21swapper/013:20:490
2153822670,6sleep10-21swapper/109:34:321
2153822670,6sleep10-21swapper/109:34:321
1406922640,6sleep333-21ksoftirqd/321:39:153
1406922640,6sleep333-21ksoftirqd/321:39:153
2079522610,8sleep10-21swapper/112:45:541
2079522610,8sleep10-21swapper/112:45:541
2079522610,8sleep10-21swapper/112:45:541
1288922600,6sleep00-21swapper/001:45:380
1288922600,6sleep00-21swapper/001:45:380
3239622580,5sleep20-21swapper/209:46:332
3239622580,5sleep20-21swapper/209:46:332
3101822580,4sleep30-21swapper/315:45:313
944422570,3sleep30-21swapper/321:29:123
944422570,3sleep30-21swapper/321:29:123
573322570,10sleep10-21swapper/121:43:071
573322570,10sleep10-21swapper/121:43:071
1543222570,4sleep015433-21latency_hist12:35:390
1543222570,4sleep015433-21latency_hist12:35:390
447022560,18sleep121-21ksoftirqd/109:32:551
447022560,18sleep121-21ksoftirqd/109:32:551
3081222550,18sleep227-21ksoftirqd/221:53:562
3081222550,18sleep227-21ksoftirqd/221:53:562
1650022520,3sleep30-21swapper/309:46:093
1650022520,3sleep30-21swapper/309:46:093
2625622510,6sleep00-21swapper/009:30:280
2625622510,6sleep00-21swapper/009:30:280
2625622510,6sleep00-21swapper/009:30:280
1513022450,6chrt21-21ksoftirqd/121:32:551
1513022450,6chrt21-21ksoftirqd/121:32:551
1223422440,4sleep012232-21cut09:29:030
1223422440,4sleep012232-21cut09:29:030
2559122390,6sleep20-21swapper/222:25:362
1969522380,6sleep10-21swapper/112:50:501
997622360,2sleep10-21swapper/101:35:501
997622360,2sleep10-21swapper/101:35:501
972122360,7sleep29723-21latency_hist10:05:292
972122360,7sleep29723-21latency_hist10:05:292
31422320,3sleep30-21swapper/315:30:493
31422320,3sleep30-21swapper/315:30:493
1094422320,6chrt27-21ksoftirqd/210:55:332
1094422320,6chrt27-21ksoftirqd/210:55:332
299922290,6sleep012-21ksoftirqd/013:35:500
299922290,6sleep012-21ksoftirqd/013:35:500
1101122280,4sleep10-21swapper/110:50:511
1640622270,4sleep015584-21/usr/sbin/munin01:35:560
1640622270,4sleep015584-21/usr/sbin/munin01:35:560
645122250,7sleep00-21swapper/010:40:530
645122250,7sleep00-21swapper/010:40:530
2221822190,1sleep07517-21kworker/0:100:25:410
695222180,7sleep06951-21cut21:29:070
695222180,7sleep06951-21cut21:29:070
1706822150,5sleep36547-21kworker/3:321:42:303
1706822150,5sleep36547-21kworker/3:321:42:303
1493222100,8sleep227-21ksoftirqd/212:30:512
1493222100,8sleep227-21ksoftirqd/212:30:512
1493222100,8sleep227-21ksoftirqd/212:30:512
865222090,7sleep2271ksoftirqd/222:35:432
865222090,7sleep2271ksoftirqd/222:35:432
122222080,5sleep20-21swapper/213:26:102
122222080,5sleep20-21swapper/213:26:102
2064122000,2sleep10-21swapper/122:50:461
1865022000,3sleep00-21swapper/011:11:010
2718921980,8sleep227-21ksoftirqd/209:40:452
2718921980,8sleep227-21ksoftirqd/209:40:452
3168521970,6sleep10-21swapper/115:50:531
1919321970,2sleep10-21swapper/109:50:561
1919321970,2sleep10-21swapper/109:50:561
973721880,3sleep09733-21tune2fs09:40:460
2026621860,17sleep227-21ksoftirqd/209:52:562
2026621860,17sleep227-21ksoftirqd/209:52:562
1397421820,2sleep10-21swapper/121:29:001
1397421820,2sleep10-21swapper/121:29:001
1627021810,5sleep00-21swapper/011:10:320
1627021810,5sleep00-21swapper/011:10:320
1339321810,20sleep121-21ksoftirqd/111:00:561
1339321810,20sleep121-21ksoftirqd/111:00:561
1247021720,6sleep10-21swapper/122:00:521
1247021720,6sleep10-21swapper/122:00:521
3056921670,7sleep130604-21sh02:53:221
3056921670,7sleep130604-21sh02:53:221
3056921670,7sleep130604-21sh02:53:221
1053421670,2sleep210535-21od14:27:582
1053421670,2sleep210535-21od14:27:582
1223521660,2sleep30-21swapper/314:51:273
1223521660,2sleep30-21swapper/314:51:273
1223521660,2sleep30-21swapper/314:51:273
1223521660,2sleep30-21swapper/314:51:273
416021640,3sleep10-21swapper/102:30:551
416021640,3sleep10-21swapper/102:30:551
2509321640,2sleep00-21swapper/008:18:080
2509321640,2sleep00-21swapper/008:18:080
1657621640,2sleep00-21swapper/021:35:460
1657621640,2sleep00-21swapper/021:35:460
1996121630,4sleep319940-21sh02:27:423
1996121630,4sleep319940-21sh02:27:423
1315721630,2sleep210946-21kworker/2:214:05:282
430021610,3sleep20-21swapper/208:39:372
430021610,3sleep20-21swapper/208:39:372
1668921600,2sleep30-21swapper/314:05:573
1668921600,2sleep30-21swapper/314:05:573
81921580,9chrt816-21rm02:52:090
81921580,9chrt816-21rm02:52:090
81921580,9chrt816-21rm02:52:090
710821580,5sleep07262-21unixbench_multi03:50:550
710821580,5sleep07262-21unixbench_multi03:50:550
3210621560,3sleep132102-21ntp_offset13:25:541
3210621560,3sleep132102-21ntp_offset13:25:541
1943621560,3sleep00-21swapper/014:15:000
1943621560,3sleep00-21swapper/014:15:000
1537221560,5sleep115370-21sort14:51:171
1537221560,5sleep115370-21sort14:51:171
1537221560,5sleep115370-21sort14:51:171
3199521550,3sleep30-21swapper/316:58:323
3199521550,3sleep30-21swapper/316:58:323
2962521550,5sleep128981-21/usr/sbin/munin15:45:581
2962521550,5sleep128981-21/usr/sbin/munin15:45:581
2830921550,4sleep028321-21expr14:53:340
2830921550,4sleep028321-21expr14:53:340
2830921550,4sleep028321-21expr14:53:340
2670021550,2sleep00-21swapper/002:55:030
2670021550,2sleep00-21swapper/002:55:030
2670021550,2sleep00-21swapper/002:55:030
2670021550,2sleep00-21swapper/002:55:030
1712021550,10sleep333-21ksoftirqd/320:37:243
1712021550,10sleep333-21ksoftirqd/320:37:243
1367221550,2sleep30-21swapper/307:39:543
1367221550,2sleep30-21swapper/307:39:543
746221530,2sleep10-21swapper/102:55:161
746221530,2sleep10-21swapper/102:55:161
746221530,2sleep10-21swapper/102:55:161
3055921530,6chrt27-21ksoftirqd/209:34:032
3055921530,6chrt27-21ksoftirqd/209:34:032
2957421530,3sleep30-21swapper/314:55:173
2957421530,3sleep30-21swapper/314:55:173
2957421530,3sleep30-21swapper/314:55:173
2415221530,5sleep024110-21execl14:33:400
2415221530,5sleep024110-21execl14:33:400
1051521530,4sleep10-21swapper/114:08:121
2933221520,8sleep333-21ksoftirqd/320:42:223
2933221520,8sleep333-21ksoftirqd/320:42:223
2933221520,8sleep333-21ksoftirqd/320:42:223
2088021520,2sleep20-21swapper/208:13:512
2088021520,2sleep20-21swapper/208:13:512
2880321510,3sleep00-21swapper/008:30:550
2880321510,3sleep00-21swapper/008:30:550
1286421510,2sleep30-21swapper/309:14:253
1286421510,2sleep30-21swapper/309:14:253
57821500,3sleep20-21swapper/206:38:272
2756521500,3sleep00-21swapper/019:25:190
2756521500,3sleep00-21swapper/019:25:190
1488821500,10sleep214889-21timerwakeupswit09:45:592
1488821500,10sleep214889-21timerwakeupswit09:45:592
1797921490,2sleep00-21swapper/018:09:500
1526421490,2sleep015219-21multi.sh14:55:410
1526421490,2sleep015219-21multi.sh14:55:410
1526421490,2sleep015219-21multi.sh14:55:410
1526421490,2sleep015219-21multi.sh14:55:410
541621480,2sleep30-21swapper/306:10:573
2523621480,2sleep20-21swapper/214:55:382
2523621480,2sleep20-21swapper/214:55:382
2523621480,2sleep20-21swapper/214:55:382
1926621480,8sleep121-21ksoftirqd/122:50:421
1926621480,8sleep121-21ksoftirqd/122:50:421
1219521480,8sleep333-21ksoftirqd/321:29:393
1219521480,8sleep333-21ksoftirqd/321:29:393
782121470,13sleep37834-21expr02:53:233
782121470,13sleep37834-21expr02:53:233
782121470,13sleep37834-21expr02:53:233
482121470,2sleep04851-21tee02:55:100
482121470,2sleep04851-21tee02:55:100
482121470,2sleep04851-21tee02:55:100
2328721470,2sleep10-21swapper/114:55:291
2328721470,2sleep10-21swapper/114:55:291
2328721470,2sleep10-21swapper/114:55:291
1964021470,3sleep00-21swapper/007:09:280
1964021470,3sleep00-21swapper/007:09:280
1498921470,2sleep20-21swapper/214:26:412
1498921470,2sleep20-21swapper/214:26:412
2107221450,3sleep00-21swapper/020:39:360
2107221450,3sleep00-21swapper/020:39:360
574921440,3sleep00-21swapper/002:55:570
574921440,3sleep00-21swapper/002:55:570
2311221440,8sleep227-21ksoftirqd/216:45:532
2311221440,8sleep227-21ksoftirqd/216:45:532
1942521420,3sleep00-21swapper/014:26:360
1942521420,3sleep00-21swapper/014:26:360
1942521420,3sleep00-21swapper/014:26:360
1855621420,3sleep30-21swapper/314:17:353
1855621420,3sleep30-21swapper/314:17:353
3045321400,5sleep030480-21sh14:53:590
3045321400,5sleep030480-21sh14:53:590
3045321400,5sleep030480-21sh14:53:590
2810021400,3sleep20-21swapper/214:23:572
2810021400,3sleep20-21swapper/214:23:572
677521390,8sleep00-21swapper/011:00:440
677521390,8sleep00-21swapper/011:00:440
1944921390,12sleep319468-21od14:55:563
1413521390,2sleep014133-21expr14:52:420
1413521390,2sleep014133-21expr14:52:420
1413521390,2sleep014133-21expr14:52:420
2899921380,7sleep012-21ksoftirqd/002:33:520
2899921380,7sleep012-21ksoftirqd/002:33:520
2490921380,3sleep324930-21wc02:54:253
2490921380,3sleep324930-21wc02:54:253
2490921380,3sleep324930-21wc02:54:253
1364621380,5chrt27-21ksoftirqd/221:38:262
1364621380,5chrt27-21ksoftirqd/221:38:262
429221370,3sleep04311-21sort02:53:430
429221370,3sleep04311-21sort02:53:430
429221370,3sleep04311-21sort02:53:430
712221360,3chrt0-21swapper/309:25:523
712221360,3chrt0-21swapper/309:25:523
661421360,6sleep06645-21sh14:55:010
661421360,6sleep06645-21sh14:55:010
661421360,6sleep06645-21sh14:55:010
661421360,6sleep06645-21sh14:55:010
2401621360,3sleep10-21swapper/102:18:281
2401621360,3sleep10-21swapper/102:18:281
1170421360,2sleep011721-21sh14:56:310
1170421360,2sleep011721-21sh14:56:310
3198621350,3chrt31968-21context102:45:300
3198621350,3chrt31968-21context102:45:300
3198621350,3chrt31968-21context102:45:300
2849121350,3chrt0-21swapper/206:00:592
2849121350,3chrt0-21swapper/206:00:592
2637021350,3sleep00-21swapper/002:19:590
1014321350,5sleep210322-21/usr/sbin/munin10:55:582
1014321350,5sleep210322-21/usr/sbin/munin10:55:582
846021340,3sleep10-21swapper/117:35:591
846021340,3sleep10-21swapper/117:35:591
1078821340,8sleep310782-21od14:51:223
1078821340,8sleep310782-21od14:51:223
1078821340,8sleep310782-21od14:51:223
949221330,8sleep333-21ksoftirqd/308:54:413
949221330,8sleep333-21ksoftirqd/308:54:413
3224521330,4sleep10-21swapper/106:05:541
3224521330,4sleep10-21swapper/106:05:541
2793021330,5sleep127933-21sh14:55:371
2793021330,5sleep127933-21sh14:55:371
2793021330,5sleep127933-21sh14:55:371
2793021330,5sleep127933-21sh14:55:371
2746421330,3sleep00-21swapper/009:02:370
2746421330,3sleep00-21swapper/009:02:370
901421320,2chrt0-21swapper/102:07:261
901421320,2chrt0-21swapper/102:07:261
2882421320,2sleep30-21swapper/314:53:533
2882421320,2sleep30-21swapper/314:53:533
2882421320,2sleep30-21swapper/314:53:533
2653321320,10sleep226541-21sh14:26:572
2653321320,10sleep226541-21sh14:26:572
1569521320,3chrt0-21swapper/002:45:340
1569521320,3chrt0-21swapper/002:45:340
829021310,5sleep28289-21latency_hist08:35:352
304821310,2sleep33052-21multi.sh14:25:553
304821310,2sleep33052-21multi.sh14:25:553
2818121310,4sleep10-21swapper/122:00:341
2818121310,4sleep10-21swapper/122:00:341
2231321310,2sleep00-21swapper/014:55:120
2231321310,2sleep00-21swapper/014:55:120
2231321310,2sleep00-21swapper/014:55:120
172921310,3sleep10-21swapper/102:55:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional