You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-03-19 - 19:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Wed Mar 19, 2025 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
265522191174,12sleep10-21swapper/107:09:521
263912189172,12sleep30-21swapper/307:07:493
264202187172,11sleep00-21swapper/007:08:100
243352183165,13sleep20-21swapper/207:05:042
270792440,0sleep00-21swapper/011:10:220
151052440,0sleep10-21swapper/111:38:301
151052440,0sleep10-21swapper/111:38:301
190992400,0sleep00-21swapper/012:32:170
102672380,0sleep30-21swapper/312:27:563
22042130,0sleep20-21swapper/209:16:112
73512100,0sleep00-21swapper/011:00:140
267369993,3cyclictest30701-21perf09:15:010
267469977,0cyclictest40-21ksoftirqd/312:39:593
267369971,3cyclictest19913-21awk09:05:000
4040260,0sleep24252-21sensors11:15:192
267469963,3cyclictest40-21ksoftirqd/310:35:023
267369960,3cyclictest10566-21awk10:45:000
25295260,0sleep30-21swapper/311:43:403
267469955,0cyclictest40-21ksoftirqd/310:15:173
267469955,0cyclictest40-21ksoftirqd/309:35:163
267469955,0cyclictest40-21ksoftirqd/308:40:153
267469955,0cyclictest40-21ksoftirqd/307:10:193
267469955,0cyclictest391ktimersoftd/310:00:123
267469955,0cyclictest391ktimersoftd/308:40:003
267469955,0cyclictest0-21swapper/310:08:563
267469954,1cyclictest40-21ksoftirqd/310:20:173
267469954,1cyclictest391ktimersoftd/308:30:153
267469953,2cyclictest40-21ksoftirqd/311:47:533
267469953,2cyclictest40-21ksoftirqd/311:32:513
267469953,2cyclictest40-21ksoftirqd/311:20:193
267469953,2cyclictest40-21ksoftirqd/311:18:073
267469953,2cyclictest40-21ksoftirqd/311:10:273
267469953,2cyclictest40-21ksoftirqd/311:00:333
267469953,2cyclictest40-21ksoftirqd/309:59:253
267469953,2cyclictest40-21ksoftirqd/309:24:153
267469953,2cyclictest40-21ksoftirqd/309:15:293
267469953,2cyclictest40-21ksoftirqd/308:55:163
267469950,5cyclictest0-21swapper/311:36:593
267469950,5cyclictest0-21swapper/311:36:583
267469950,5cyclictest0-21swapper/310:48:053
267469950,5cyclictest0-21swapper/309:25:173
267469950,4cyclictest27786-21apt-get08:15:113
267469950,4cyclictest14041-21ssh12:30:123
267469950,4cyclictest0-21swapper/312:15:123
267469950,4cyclictest0-21swapper/310:55:133
267419955,0cyclictest0-21swapper/208:35:172
267419950,5cyclictest0-21swapper/212:20:132
267419950,5cyclictest0-21swapper/209:10:132
267419950,0cyclictest0-21swapper/210:36:042
267379951,3cyclictest0-21swapper/110:40:091
267379951,2cyclictest0-21swapper/111:55:121
267379951,2cyclictest0-21swapper/110:15:111
267379951,2cyclictest0-21swapper/108:10:151
267379951,2cyclictest0-21swapper/108:00:131
267379950,5cyclictest0-21swapper/112:31:511
267379950,5cyclictest0-21swapper/110:45:541
267379950,5cyclictest0-21swapper/110:27:091
267379950,5cyclictest0-21swapper/109:22:151
267369955,0cyclictest0-21swapper/007:55:170
267369954,1cyclictest3-21ksoftirqd/010:15:110
267369953,1cyclictest3-21ksoftirqd/011:30:080
267369951,1cyclictest81rcu_preempt12:15:120
267369950,5cyclictest0-21swapper/010:55:070
4948240,0sleep10-21swapper/112:07:571
267469944,0cyclictest40-21ksoftirqd/311:55:133
267469944,0cyclictest40-21ksoftirqd/311:50:083
267469944,0cyclictest40-21ksoftirqd/310:30:093
267469944,0cyclictest40-21ksoftirqd/309:45:023
267469944,0cyclictest40-21ksoftirqd/309:30:123
267469944,0cyclictest40-21ksoftirqd/308:05:203
267469944,0cyclictest40-21ksoftirqd/307:40:153
267469944,0cyclictest40-21ksoftirqd/307:20:013
267469944,0cyclictest391ktimersoftd/307:55:153
267469942,2cyclictest40-21ksoftirqd/312:05:413
267469942,2cyclictest40-21ksoftirqd/311:25:083
267469942,2cyclictest40-21ksoftirqd/311:05:223
267469942,2cyclictest40-21ksoftirqd/310:25:423
267469942,2cyclictest40-21ksoftirqd/309:50:083
267469942,2cyclictest40-21ksoftirqd/307:35:093
267469942,2cyclictest391ktimersoftd/310:50:023
267469941,3cyclictest391ktimersoftd/307:15:003
267469940,4cyclictest20369-21apt-get08:00:143
267469940,4cyclictest0-21swapper/312:20:143
267469940,4cyclictest0-21swapper/312:00:113
267469940,4cyclictest0-21swapper/310:40:033
267469940,4cyclictest0-21swapper/310:10:083
267469940,4cyclictest0-21swapper/309:10:083
267469940,4cyclictest0-21swapper/309:00:323
267469940,4cyclictest0-21swapper/308:25:133
267469940,4cyclictest0-21swapper/308:25:123
267469940,4cyclictest0-21swapper/307:30:173
267469940,3cyclictest8393-21ssh12:10:003
267469940,3cyclictest3355-21df_inode07:25:133
267469940,3cyclictest14852-21ssh09:40:013
267469940,0cyclictest0-21swapper/307:51:523
267419944,0cyclictest0-21swapper/211:53:552
267419944,0cyclictest0-21swapper/211:45:022
267419944,0cyclictest0-21swapper/211:39:232
267419944,0cyclictest0-21swapper/211:39:232
267419944,0cyclictest0-21swapper/210:58:372
267419944,0cyclictest0-21swapper/209:46:012
267419944,0cyclictest0-21swapper/209:25:392
267419942,1cyclictest17661-21/usr/sbin/munin09:00:142
267419942,1cyclictest11107-21ssh10:11:252
267419940,4cyclictest27869-21df09:30:092
267419940,4cyclictest18515-21apt-get12:15:112
267419940,4cyclictest15740-21awk12:30:192
267419940,4cyclictest0-21swapper/212:10:092
267419940,4cyclictest0-21swapper/212:06:062
267419940,4cyclictest0-21swapper/211:55:112
267419940,4cyclictest0-21swapper/211:07:222
267419940,4cyclictest0-21swapper/210:26:112
267419940,4cyclictest0-21swapper/209:40:142
267419940,4cyclictest0-21swapper/209:36:002
267419940,4cyclictest0-21swapper/209:21:362
267419940,3cyclictest32153-21/usr/sbin/munin11:30:142
267419940,3cyclictest2782-21chrt08:30:072
267419940,3cyclictest2753-21sed10:40:172
267419940,3cyclictest25405-21sh12:00:242
267419940,3cyclictest23574-21sh12:35:012
267419940,3cyclictest18042-21ssh11:40:122
267419940,3cyclictest16594-21sh11:21:342
267419940,3cyclictest15336-21latency_hist07:50:002
267419940,2cyclictest6149-21sh09:52:072
267419940,2cyclictest0-21swapper/210:17:342
267419940,1cyclictest22836-21/usr/sbin/munin08:05:222
267419940,1cyclictest19884-21perl10:50:102
267379944,0cyclictest0-21swapper/111:30:171
267379944,0cyclictest0-21swapper/111:06:341
267379941,2cyclictest0-21swapper/111:00:121
267379941,2cyclictest0-21swapper/110:50:071
267379941,2cyclictest0-21swapper/110:20:101
267379941,2cyclictest0-21swapper/109:45:011
267379941,2cyclictest0-21swapper/108:20:121
267379941,2cyclictest0-21swapper/108:15:121
267379941,2cyclictest0-21swapper/107:55:111
267379941,2cyclictest0-21swapper/107:25:121
267379941,2cyclictest0-21swapper/107:15:131
267379941,1cyclictest0-21swapper/111:10:121
267379941,1cyclictest0-21swapper/108:05:121
267379940,2cyclictest0-21swapper/111:50:071
267379940,0cyclictest0-21swapper/110:38:031
267379940,0cyclictest0-21swapper/109:58:471
267379940,0cyclictest0-21swapper/109:44:041
267369944,0cyclictest3-21ksoftirqd/008:20:000
267369944,0cyclictest0-21swapper/011:17:140
267369943,1cyclictest32520-21latency_hist08:25:010
267369942,2cyclictest30997-21snmpd11:37:310
267369942,2cyclictest30997-21snmpd11:37:310
267369942,1cyclictest30997-21snmpd12:36:040
267369940,4cyclictest21662-21ssh11:24:460
267369940,4cyclictest17476-21ssh11:05:330
267369940,4cyclictest16529-21ssh10:31:010
267369940,4cyclictest0-21swapper/012:27:360
267369940,4cyclictest0-21swapper/012:06:210
267369940,4cyclictest0-21swapper/011:55:110
267369940,4cyclictest0-21swapper/011:50:480
267369940,4cyclictest0-21swapper/011:45:430
267369940,4cyclictest0-21swapper/010:07:140
267369940,4cyclictest0-21swapper/009:56:450
267369940,4cyclictest0-21swapper/009:47:070
267369940,4cyclictest0-21swapper/009:43:230
267369940,4cyclictest0-21swapper/009:35:020
267369940,4cyclictest0-21swapper/009:20:420
267369940,4cyclictest0-21swapper/008:00:130
267369940,4cyclictest0-21swapper/007:50:150
267369940,4cyclictest0-21swapper/007:40:130
267369940,4cyclictest0-21swapper/007:35:090
267369940,3cyclictest29840-21sed07:15:010
267369940,3cyclictest27857-21rm12:01:570
267369940,3cyclictest27314-21ssh09:30:080
267369940,3cyclictest23155-21ssh10:35:020
267369940,3cyclictest22836-21/usr/sbin/munin08:05:220
267369940,3cyclictest19837-21perl10:50:090
267369940,3cyclictest18849-21ssh09:25:010
267369940,3cyclictest15766-21ssh10:48:090
267369940,3cyclictest1161-21/usr/sbin/munin09:50:170
267369940,2cyclictest81rcu_preempt10:00:120
267369940,1cyclictest8410-21cron09:20:000
267369940,1cyclictest15176-21idleruntime-cro07:50:000
267369940,0cyclictest0-21swapper/010:10:020
2141240,0sleep10-21swapper/107:20:201
1766240,0sleep00-21swapper/012:22:370
267469933,0cyclictest0-21swapper/307:45:103
267469931,1cyclictest0-21swapper/308:50:143
267469930,3cyclictest0-21swapper/309:08:293
267469930,3cyclictest0-21swapper/308:45:133
267469930,3cyclictest0-21swapper/308:20:113
267469930,3cyclictest0-21swapper/308:10:023
267419932,1cyclictest27096-21cut07:10:142
267419931,2cyclictest57250irq/125-eth011:25:032
267419931,1cyclictest0-21swapper/210:30:072
267419931,1cyclictest0-21swapper/208:40:122
267419931,1cyclictest0-21swapper/208:20:122
267419930,3cyclictest30997-21snmpd12:25:172
267419930,3cyclictest30163-21basename07:15:122
267419930,3cyclictest27964-21cat08:15:122
267419930,3cyclictest21411-21ssh10:00:092
267419930,3cyclictest20437-21/usr/sbin/munin08:00:132
267419930,3cyclictest12698-21/usr/sbin/munin08:50:152
267419930,3cyclictest0-21swapper/211:10:102
267419930,3cyclictest0-21swapper/211:00:122
267419930,3cyclictest0-21swapper/210:45:042
267419930,3cyclictest0-21swapper/209:55:022
267419930,3cyclictest0-21swapper/209:05:292
267419930,3cyclictest0-21swapper/208:45:132
267419930,3cyclictest0-21swapper/208:25:022
267419930,3cyclictest0-21swapper/208:25:012
267419930,3cyclictest0-21swapper/208:10:142
267419930,3cyclictest0-21swapper/207:55:112
267419930,3cyclictest0-21swapper/207:50:152
267419930,3cyclictest0-21swapper/207:40:022
267419930,3cyclictest0-21swapper/207:35:012
267419930,3cyclictest0-21swapper/207:30:162
267419930,3cyclictest0-21swapper/207:25:122
267419930,3cyclictest0-21swapper/207:20:012
267419930,1cyclictest31272-21sh10:05:022
267419930,1cyclictest27388-21ssh10:20:062
267419930,1cyclictest15173-21/usr/sbin/munin08:55:142
267379933,0cyclictest0-21swapper/112:01:511
267379933,0cyclictest0-21swapper/108:45:161
267379932,1cyclictest0-21swapper/110:30:081
267379931,1cyclictest0-21swapper/112:35:111
267379931,1cyclictest0-21swapper/112:20:131
267379931,1cyclictest0-21swapper/112:15:111
267379931,1cyclictest0-21swapper/111:40:131
267379931,1cyclictest0-21swapper/111:15:101
267379931,1cyclictest0-21swapper/110:55:141
267379931,1cyclictest0-21swapper/110:00:131
267379931,1cyclictest0-21swapper/109:50:081
267379931,1cyclictest0-21swapper/109:35:121
267379931,1cyclictest0-21swapper/109:15:131
267379931,1cyclictest0-21swapper/108:50:131
267379931,1cyclictest0-21swapper/108:40:131
267379931,1cyclictest0-21swapper/108:35:111
267379931,1cyclictest0-21swapper/108:30:151
267379931,1cyclictest0-21swapper/108:25:121
267379931,1cyclictest0-21swapper/108:25:121
267379931,1cyclictest0-21swapper/107:35:091
267379931,1cyclictest0-21swapper/107:30:161
267379930,2cyclictest105852sleep110:11:061
267379930,2cyclictest0-21swapper/111:20:161
267379930,2cyclictest0-21swapper/109:00:101
267369932,1cyclictest30997-21snmpd12:10:070
267369932,1cyclictest30997-21snmpd11:25:010
267369931,2cyclictest30997-21snmpd08:45:220
267369931,2cyclictest30997-21snmpd07:30:400
267369930,3cyclictest2893-21basename07:25:130
267369930,3cyclictest25311-21apt-get08:10:140
267369930,3cyclictest15693-21df08:55:150
267369930,3cyclictest0-21swapper/011:40:070
267369930,3cyclictest0-21swapper/010:25:160
267369930,3cyclictest0-21swapper/010:20:010
267369930,3cyclictest0-21swapper/009:05:220
267369930,3cyclictest0-21swapper/008:50:130
267369930,3cyclictest0-21swapper/008:40:120
267369930,3cyclictest0-21swapper/008:35:100
267369930,3cyclictest0-21swapper/008:30:140
267369930,3cyclictest0-21swapper/008:25:130
267369930,3cyclictest0-21swapper/008:25:120
267369930,3cyclictest0-21swapper/007:20:010
267369930,3cyclictest0-21swapper/007:10:130
21784230,0sleep10-21swapper/109:26:511
267379922,0cyclictest0-21swapper/111:26:141
267379921,1cyclictest151632chrt08:55:071
267379921,1cyclictest0-21swapper/112:10:101
267379920,2cyclictest8155-21chrt12:26:341
267379920,2cyclictest20869-21taskset09:10:051
267379920,2cyclictest0-21swapper/107:10:141
267379920,1cyclictest20170-21chrt09:05:461
267379920,1cyclictest0-21swapper/111:46:351
267379920,1cyclictest0-21swapper/110:05:431
267379920,1cyclictest0-21swapper/109:30:091
267379920,1cyclictest0-21swapper/107:50:151
267379920,1cyclictest0-21swapper/107:45:111
267379920,1cyclictest0-21swapper/107:40:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional