You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-29 - 00:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Tue Oct 28, 2025 12:44:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19432197181,12sleep00-21swapper/007:07:150
19392190172,13sleep10-21swapper/107:07:121
18842181163,13sleep30-21swapper/307:06:283
323562171155,12sleep20-21swapper/207:05:072
304232690,0sleep00-21swapper/010:13:500
181272420,0sleep10-21swapper/111:44:581
156562420,0sleep30-21swapper/311:27:033
11142420,0sleep30-21swapper/311:04:023
3902410,0sleep00-21swapper/011:51:340
81462400,1sleep28147-21perf07:20:002
205712120,1sleep1211ktimersoftd/109:36:031
255382110,0sleep10-21swapper/111:48:151
235899102,4cyclictest4559-21awk09:45:010
23589992,3cyclictest23186-21munin-run07:50:010
8440280,1sleep08437-21perf09:15:010
23589981,3cyclictest6383-21munin-run09:30:000
23589970,3cyclictest10350-21perf11:24:590
23709960,5cyclictest0-21swapper/309:50:123
23679961,4cyclictest0-21swapper/211:20:112
23629960,6cyclictest0-21swapper/111:59:431
23589960,3cyclictest18206-21munin-run08:45:000
23589960,3cyclictest18192-21munin-run11:45:000
22764260,1sleep1236299cyclictest11:15:001
19673260,0sleep20-21swapper/212:17:382
12232260,0sleep10-21swapper/109:16:051
23709950,5cyclictest2451-21apt-get07:10:133
23679955,0cyclictest0-21swapper/210:26:542
23679955,0cyclictest0-21swapper/210:14:152
23679950,5cyclictest0-21swapper/212:09:562
23679950,0cyclictest0-21swapper/210:07:332
23629955,0cyclictest0-21swapper/111:27:071
23629951,2cyclictest0-21swapper/110:05:161
23629950,5cyclictest0-21swapper/112:26:191
23629950,5cyclictest0-21swapper/111:51:341
23629950,5cyclictest0-21swapper/111:34:321
23629950,5cyclictest0-21swapper/111:07:021
23629950,5cyclictest0-21swapper/110:47:401
23629950,5cyclictest0-21swapper/109:52:241
23589955,0cyclictest0-21swapper/010:35:030
23709944,0cyclictest0-21swapper/311:20:383
23709944,0cyclictest0-21swapper/310:33:043
23709943,1cyclictest3574-21/usr/sbin/munin08:15:173
23709942,1cyclictest711-21runrttasks08:10:113
23709942,1cyclictest0-21swapper/308:30:133
23709941,3cyclictest9610-21tr10:35:203
23709941,3cyclictest55950irq/130-eth012:02:153
23709941,3cyclictest495-21nscd10:17:123
23709941,1cyclictest0-21swapper/309:30:113
23709940,4cyclictest8962-21apt-get09:15:143
23709940,4cyclictest7739-21ssh10:50:223
23709940,4cyclictest25142-21memory12:20:173
23709940,4cyclictest23599-21cat11:15:133
23709940,4cyclictest0-21swapper/312:12:353
23709940,4cyclictest0-21swapper/311:45:133
23709940,4cyclictest0-21swapper/311:10:143
23709940,4cyclictest0-21swapper/310:58:223
23709940,4cyclictest0-21swapper/310:40:133
23709940,4cyclictest0-21swapper/310:01:463
23709940,4cyclictest0-21swapper/309:37:583
23709940,4cyclictest0-21swapper/309:29:353
23709940,4cyclictest0-21swapper/307:38:533
23709940,3cyclictest8317-21find11:40:003
23709940,3cyclictest5348-21ssh11:54:403
23709940,3cyclictest5153-21tr09:45:143
23709940,3cyclictest4964-21munin-node07:15:013
23709940,3cyclictest3618-21sh09:12:033
23709940,3cyclictest32716-21sh09:42:233
23709940,3cyclictest32606-21sh09:58:433
23709940,3cyclictest31459-21sh11:35:153
23709940,3cyclictest21784-21hddtemp_smartct07:45:153
23709940,3cyclictest20893-21sh10:09:103
23709940,3cyclictest20585-21rm11:30:023
23709940,3cyclictest18806-21sh09:20:023
23709940,3cyclictest18571-21apt-config08:45:013
23709940,3cyclictest14485-21rm12:31:063
23709940,3cyclictest10850-21apt-get07:25:133
23709940,3cyclictest0-21swapper/310:45:123
23709940,2cyclictest0-21swapper/310:20:323
23709940,1cyclictest7836-21ntp_states11:55:183
23709940,1cyclictest3251-21dpkg11:05:133
23709940,1cyclictest28534-21/usr/sbin/munin09:05:233
23709940,1cyclictest23909-21irqstats10:10:173
23709940,1cyclictest21083-21/usr/sbin/munin12:35:133
23709940,1cyclictest14590-21ntp_states07:30:203
23709940,1cyclictest1417-21/usr/sbin/munin12:25:233
23709940,1cyclictest13476-21/usr/sbin/munin08:35:183
23709940,1cyclictest0-21swapper/310:25:153
23679944,0cyclictest0-21swapper/212:22:262
23679944,0cyclictest0-21swapper/211:40:242
23679944,0cyclictest0-21swapper/211:15:132
23679941,3cyclictest714-21snmpd11:50:142
23679941,3cyclictest3688-21ntpq07:10:182
23679941,3cyclictest0-21swapper/209:56:392
23679941,3cyclictest0-21swapper/208:54:062
23679941,2cyclictest0-21swapper/208:40:122
23679940,4cyclictest30535-21ssh10:30:142
23679940,4cyclictest16933-21diskstats12:00:132
23679940,4cyclictest0-21swapper/212:36:152
23679940,4cyclictest0-21swapper/212:33:302
23679940,4cyclictest0-21swapper/212:10:132
23679940,4cyclictest0-21swapper/211:30:132
23679940,4cyclictest0-21swapper/211:26:262
23679940,4cyclictest0-21swapper/211:12:192
23679940,4cyclictest0-21swapper/211:05:132
23679940,4cyclictest0-21swapper/210:55:142
23679940,4cyclictest0-21swapper/210:00:152
23679940,4cyclictest0-21swapper/209:45:212
23679940,4cyclictest0-21swapper/209:35:132
23679940,4cyclictest0-21swapper/209:32:362
23679940,4cyclictest0-21swapper/208:55:142
23679940,4cyclictest0-21swapper/207:30:152
23679940,3cyclictest7933-21ssh10:18:472
23679940,3cyclictest7128-21ssh09:14:132
23679940,3cyclictest535-21users11:35:212
23679940,3cyclictest3499-21apt-config08:15:002
23679940,3cyclictest32530-21taskset11:03:232
23679940,3cyclictest30366-21/usr/sbin/munin09:25:182
23679940,3cyclictest30360-21ssh10:45:312
23679940,3cyclictest20344-21ntpq11:45:192
23679940,3cyclictest1420-21diskstats08:10:132
23679940,3cyclictest10245-21ssh10:35:252
23679940,3cyclictest10036-21ssh09:15:182
23679940,1cyclictest24196-21ssh09:54:452
23679940,0cyclictest0-21swapper/210:20:132
23679940,0cyclictest0-21swapper/207:26:382
23629944,0cyclictest0-21swapper/112:17:281
23629944,0cyclictest0-21swapper/110:36:281
23629944,0cyclictest0-21swapper/110:19:181
23629944,0cyclictest0-21swapper/109:02:371
23629942,1cyclictest5292sleep111:35:211
23629942,1cyclictest28538-21chrt10:12:381
23629942,1cyclictest17930-21chrt08:41:421
23629942,1cyclictest0-21swapper/110:55:151
23629942,1cyclictest0-21swapper/107:20:121
23629941,2cyclictest0-21swapper/112:35:121
23629941,2cyclictest0-21swapper/112:20:131
23629941,2cyclictest0-21swapper/112:10:121
23629941,2cyclictest0-21swapper/112:05:001
23629941,2cyclictest0-21swapper/109:25:151
23629941,2cyclictest0-21swapper/108:25:131
23629941,1cyclictest0-21swapper/109:45:131
23629940,4cyclictest0-21swapper/108:07:371
23589944,0cyclictest0-21swapper/012:11:300
23589944,0cyclictest0-21swapper/012:01:540
23589944,0cyclictest0-21swapper/011:35:170
23589944,0cyclictest0-21swapper/011:06:190
23589944,0cyclictest0-21swapper/010:52:120
23589944,0cyclictest0-21swapper/009:20:580
23589943,1cyclictest16694-21kworker/0:108:23:250
23589942,1cyclictest714-21snmpd07:35:240
23589941,2cyclictest0-21swapper/009:45:130
23589941,1cyclictest156572sh11:27:040
23589940,4cyclictest30774-21diskmemload09:16:310
23589940,4cyclictest22435-21ssh09:53:390
23589940,4cyclictest0-21swapper/012:35:120
23589940,4cyclictest0-21swapper/012:31:270
23589940,4cyclictest0-21swapper/012:05:130
23589940,4cyclictest0-21swapper/011:57:100
23589940,4cyclictest0-21swapper/011:45:200
23589940,4cyclictest0-21swapper/011:00:140
23589940,4cyclictest0-21swapper/010:45:500
23589940,4cyclictest0-21swapper/010:31:530
23589940,4cyclictest0-21swapper/010:25:140
23589940,4cyclictest0-21swapper/009:55:140
23589940,4cyclictest0-21swapper/009:00:110
23589940,4cyclictest0-21swapper/007:20:130
23589940,3cyclictest8513-21ntp_states09:30:180
23589940,3cyclictest3147-21perl07:10:160
23589940,3cyclictest30774-21diskmemload10:42:140
23589940,3cyclictest26333-21sh12:20:340
23589940,3cyclictest22290-21ssh11:30:190
23589940,3cyclictest11948-21ntp_states10:20:200
23589940,0cyclictest0-21swapper/009:39:450
23709933,0cyclictest0-21swapper/309:00:043
23709931,1cyclictest711-21runrttasks07:20:133
23709931,1cyclictest0-21swapper/308:05:123
23709930,3cyclictest8806-21ls08:25:143
23709930,3cyclictest26366-21rm12:05:063
23709930,3cyclictest184782sleep307:40:113
23709930,3cyclictest16305-21tr08:40:143
23709930,3cyclictest14006-21apt-get12:15:113
23709930,3cyclictest0-21swapper/308:55:133
23709930,3cyclictest0-21swapper/308:50:143
23709930,3cyclictest0-21swapper/308:20:123
23709930,3cyclictest0-21swapper/308:00:133
23709930,3cyclictest0-21swapper/307:55:023
23709930,3cyclictest0-21swapper/307:50:123
23679932,1cyclictest9118-21tune2fs07:20:152
23679932,1cyclictest714-21snmpd08:45:092
23679932,1cyclictest21927-21awk07:45:162
23679931,1cyclictest0-21swapper/208:00:142
23679931,1cyclictest0-21swapper/207:50:112
23679930,3cyclictest5386-21ssh10:50:042
23679930,3cyclictest28538-21apt-get09:05:142
23679930,3cyclictest27193-21taskset09:40:022
23679930,3cyclictest26139-21aten2_r5power_p09:00:112
23679930,3cyclictest25891-21apt-get07:55:142
23679930,3cyclictest18991-21ssh09:20:092
23679930,3cyclictest11539-21diskstats08:30:152
23679930,3cyclictest0-21swapper/212:25:012
23679930,3cyclictest0-21swapper/211:55:062
23679930,3cyclictest0-21swapper/210:40:042
23679930,3cyclictest0-21swapper/208:35:142
23679930,3cyclictest0-21swapper/208:25:142
23679930,3cyclictest0-21swapper/208:20:122
23679930,3cyclictest0-21swapper/208:05:122
23679930,3cyclictest0-21swapper/207:40:132
23679930,3cyclictest0-21swapper/207:35:142
23629933,0cyclictest0-21swapper/111:00:201
23629933,0cyclictest0-21swapper/108:16:031
23629932,1cyclictest29873-21chrt09:05:211
23629932,1cyclictest28904-21chrt08:00:161
23629932,1cyclictest2436-21chrt07:10:041
23629932,1cyclictest18471-21chrt07:40:061
23629932,1cyclictest13307-21chrt07:30:041
23629932,1cyclictest12888-21chrt08:30:381
23629932,1cyclictest12721-21chrt07:25:321
23629932,1cyclictest0-21swapper/109:10:131
23629932,1cyclictest0-21swapper/108:10:121
23629931,1cyclictest5570-21taskset10:50:101
23629931,1cyclictest20338-21chrt10:25:081
23629931,1cyclictest178572chrt10:40:051
23629931,1cyclictest0-21swapper/112:00:121
23629931,1cyclictest0-21swapper/111:15:131
23629931,1cyclictest0-21swapper/110:20:151
23629931,1cyclictest0-21swapper/110:00:161
23629931,1cyclictest0-21swapper/109:30:021
23629931,1cyclictest0-21swapper/108:50:121
23629931,1cyclictest0-21swapper/108:45:111
23629931,1cyclictest0-21swapper/108:35:141
23629931,1cyclictest0-21swapper/108:20:111
23629931,1cyclictest0-21swapper/107:55:131
23629931,1cyclictest0-21swapper/107:50:111
23629931,1cyclictest0-21swapper/107:45:121
23629931,1cyclictest0-21swapper/107:15:131
23629930,2cyclictest0-21swapper/112:30:111
23629930,1cyclictest0-21swapper/109:20:121
23589933,0cyclictest16694-21kworker/0:110:55:060
23589933,0cyclictest16694-21kworker/0:108:35:000
23589933,0cyclictest16694-21kworker/0:108:15:090
23589933,0cyclictest16694-21kworker/0:107:40:100
23589932,1cyclictest1810-21grep12:25:150
23589932,1cyclictest13156-21ssh11:10:080
23589932,1cyclictest1107-21expr08:10:130
23589931,1cyclictest24331-21if_eth111:15:160
23589931,1cyclictest0-21swapper/008:55:130
23589930,3cyclictest5049-21sendmail_mailtr07:15:020
23589930,3cyclictest483-21taskset10:15:110
23589930,3cyclictest3454-21df_inode10:00:170
23589930,3cyclictest24157-21perl07:50:150
23589930,3cyclictest13724-21rm12:15:000
23589930,3cyclictest11533-21df_inode08:30:140
23589930,3cyclictest0-21swapper/010:05:170
23589930,3cyclictest0-21swapper/009:05:150
23589930,3cyclictest0-21swapper/008:50:050
23589930,3cyclictest0-21swapper/008:45:010
23589930,3cyclictest0-21swapper/008:05:120
23589930,3cyclictest0-21swapper/008:00:130
23589930,3cyclictest0-21swapper/007:55:130
23589930,3cyclictest0-21swapper/007:30:140
23589930,3cyclictest0-21swapper/007:25:130
23589930,2cyclictest8537-21/usr/sbin/munin08:25:130
23629921,1cyclictest761-21chrt11:20:091
23629921,1cyclictest272242sleep109:40:041
23629921,1cyclictest25177-21chrt09:55:141
23629920,1cyclictest0-21swapper/110:30:151
23629920,1cyclictest0-21swapper/108:55:141
23629920,1cyclictest0-21swapper/107:35:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional