You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-05 - 05:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5.osadl.org (updated Fri Dec 05, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6308993030,0cyclictest29014-21apt-get22:15:101
630899300,29cyclictest11467-21sh23:55:001
630499300,29cyclictest609-21dbus-daemon22:30:010
6310992927,1cyclictest303-21systemd-journal00:35:013
6310992927,1cyclictest0-21swapper/321:20:013
631099290,28cyclictest3600-21apt-get22:30:163
631099290,1cyclictest9432-21apt-get23:50:123
631099290,1cyclictest8278-21apt-get22:40:153
631099290,1cyclictest0-21swapper/320:40:013
630999290,28cyclictest19676-21cron23:05:002
630999290,28cyclictest0-21swapper/223:35:142
630999290,28cyclictest0-21swapper/222:55:142
630899290,28cyclictest0-21swapper/123:40:131
630899290,1cyclictest14397-21cron21:45:001
630499290,28cyclictest5368-21apt-get21:25:150
630499290,28cyclictest0-21swapper/021:30:180
630499290,28cyclictest0-21swapper/020:00:140
630499290,28cyclictest0-21swapper/019:20:110
6310992827,1cyclictest17019-21apt-get21:50:123
631099280,27cyclictest0-21swapper/321:05:153
631099280,0cyclictest29935-21apt-get20:00:023
6309992827,1cyclictest23793-21apt-get20:55:142
6309992826,1cyclictest16471-21sendmail-msp20:40:002
630999280,28cyclictest0-21swapper/223:45:032
630999280,28cyclictest0-21swapper/219:45:142
630999280,27cyclictest9767-21apt-get20:25:122
630999280,0cyclictest0-21swapper/222:40:152
630899280,27cyclictest0-21swapper/120:45:121
6304992828,0cyclictest0-21swapper/000:25:170
6304992827,1cyclictest10938-21sendmail-msp19:20:000
630499280,27cyclictest0-21swapper/022:20:130
6310992727,0cyclictest0-21swapper/319:10:133
6310992726,1cyclictest27889-21apt-get19:55:153
6310992725,1cyclictest27617-21perf19:55:013
6309992727,0cyclictest5923-21apt-get22:35:142
6309992727,0cyclictest0-21swapper/221:15:132
6309992727,0cyclictest0-21swapper/220:05:012
6308992727,0cyclictest0-21swapper/121:25:141
6308992727,0cyclictest0-21swapper/120:30:181
6308992726,1cyclictest19402-21apt-get21:55:161
6308992724,2cyclictest373-21perf21:15:001
630899270,0cyclictest0-21swapper/120:10:141
6304992727,0cyclictest0-21swapper/023:35:000
6304992727,0cyclictest0-21swapper/022:05:140
630499270,0cyclictest30784-21apt-get21:10:110
6310992626,0cyclictest0-21swapper/321:45:153
6310992625,1cyclictest0-21swapper/322:10:143
6309992625,1cyclictest28241-21munin-run00:30:022
6308992626,0cyclictest20914-21apt-get19:40:121
6308992626,0cyclictest0-21swapper/123:05:141
6304992626,0cyclictest0-21swapper/021:40:120
6309992525,0cyclictest28462-21apt-get21:05:152
6309992525,0cyclictest0-21swapper/223:50:122
6309992525,0cyclictest0-21swapper/220:20:132
6308992525,0cyclictest0-21swapper/123:55:141
6308992525,0cyclictest0-21swapper/123:15:131
6308992525,0cyclictest0-21swapper/121:35:111
6308992525,0cyclictest0-21swapper/119:25:111
6308992525,0cyclictest0-21swapper/119:15:121
6308992525,0cyclictest0-21swapper/119:15:011
6308992525,0cyclictest0-21swapper/100:00:141
6304992525,0cyclictest0-21swapper/023:25:130
6304992525,0cyclictest0-21swapper/021:20:130
6310992424,0cyclictest22303-21apt-get23:10:173
6310992424,0cyclictest0-21swapper/322:50:133
6310992424,0cyclictest0-21swapper/319:35:133
6310992424,0cyclictest0-21swapper/319:35:133
6310992424,0cyclictest0-21swapper/319:25:123
6308992424,0cyclictest0-21swapper/123:35:041
6308992424,0cyclictest0-21swapper/123:00:131
6308992424,0cyclictest0-21swapper/119:35:131
6308992424,0cyclictest0-21swapper/119:35:121
6308992424,0cyclictest0-21swapper/100:35:031
6304992424,0cyclictest0-21swapper/023:20:130
6304992424,0cyclictest0-21swapper/020:15:130
6309992323,0cyclictest0-21swapper/222:30:162
6308992323,0cyclictest0-21swapper/100:30:141
6308992321,1cyclictest10315-21awk22:45:011
6308992320,2cyclictest1235-21idleruntime-cro23:35:011
6304992323,0cyclictest0-21swapper/022:55:150
6310992222,0cyclictest0-21swapper/322:55:153
6310992222,0cyclictest0-21swapper/300:10:143
6310992221,0cyclictest0-21swapper/323:35:143
6309992222,0cyclictest0-21swapper/222:10:132
6309992221,1cyclictest626-21gdbus00:35:022
6309992221,0cyclictest14698-21apt-get21:45:162
630899223,4cyclictest22-21ksoftirqd/121:15:171
6308992222,0cyclictest22-21ksoftirqd/121:33:111
6308992222,0cyclictest11231-21apt-get19:20:111
6308992222,0cyclictest0-21swapper/122:25:141
6308992222,0cyclictest0-21swapper/120:00:151
6304992220,1cyclictest0-21swapper/019:15:010
6304992220,1cyclictest0-21swapper/000:00:000
6310992121,0cyclictest0-21swapper/321:10:123
6309992121,0cyclictest0-21swapper/221:50:122
6309992121,0cyclictest0-21swapper/219:55:152
6309992121,0cyclictest0-21swapper/219:10:132
6308992121,0cyclictest0-21swapper/121:20:121
6304992118,2cyclictest20161-21snmpd21:55:150
6310992020,0cyclictest0-21swapper/320:55:143
6309992018,1cyclictest609-21dbus-daemon23:45:012
6308992020,0cyclictest10601-21apt-get22:45:141
6308992020,0cyclictest0-21swapper/100:05:121
6308992017,2cyclictest23239-21apt-get19:45:141
6304992020,0cyclictest14159-21perf20:35:010
6304992020,0cyclictest0-21swapper/023:10:170
6304992020,0cyclictest0-21swapper/020:20:140
6309991919,0cyclictest12924-21apt-get22:50:132
6309991919,0cyclictest0-21swapper/219:55:022
6309991919,0cyclictest0-21swapper/219:35:122
6309991919,0cyclictest0-21swapper/219:35:122
6308991919,0cyclictest0-21swapper/123:10:171
6308991919,0cyclictest0-21swapper/122:00:111
6304991919,0cyclictest0-21swapper/020:55:130
6304991919,0cyclictest0-21swapper/000:30:130
6304991919,0cyclictest0-21swapper/000:20:140
6309991818,0cyclictest0-21swapper/219:25:122
6309991817,0cyclictest19183-21apt-get00:10:142
630999180,5cyclictest0-21swapper/223:10:162
6308991818,0cyclictest0-21swapper/100:20:141
6304991818,0cyclictest0-21swapper/020:10:140
6304991818,0cyclictest0-21swapper/000:35:020
6304991816,1cyclictest29918-21cron20:00:000
6310991717,0cyclictest0-21swapper/323:25:123
630899170,16cyclictest583-21cpuspeed_turbos20:05:141
6304991717,0cyclictest0-21swapper/023:00:130
6310991616,0cyclictest7444-21apt-get20:20:143
631099160,15cyclictest6132-21ntp_states20:15:193
631099160,15cyclictest22380-21sed22:00:143
631099160,15cyclictest14525-21/usr/sbin/munin21:45:003
6309991616,0cyclictest0-21swapper/221:10:122
6309991614,1cyclictest20392-21mailstats20:45:202
6309991613,3cyclictest0-21swapper/221:38:232
6308991616,0cyclictest0-21swapper/121:00:181
6308991616,0cyclictest0-21swapper/120:50:151
6308991616,0cyclictest0-21swapper/100:10:141
6304991615,1cyclictest63250irq/126-eno119:45:230
6304991614,1cyclictest0-21swapper/000:17:450
6310991513,1cyclictest21323-21latency_hist20:49:593
631099150,1cyclictest26602-21diskstats00:25:183
631099150,1cyclictest16489-21apt-config00:05:013
631099150,1cyclictest0-21swapper/323:45:033
631099150,14cyclictest0-21swapper/320:06:253
6309991513,1cyclictest1264-21apt-get22:25:132
630999150,15cyclictest31115-21cat22:19:592
6308991515,0cyclictest0-21swapper/122:05:131
630899150,1cyclictest0-21swapper/120:25:201
630899150,14cyclictest0-21swapper/121:05:171
6304991515,0cyclictest18266-21apt-get19:35:130
6304991515,0cyclictest18266-21apt-get19:35:130
6304991515,0cyclictest0-21swapper/000:00:150
6304991513,1cyclictest16498-21munin-run20:40:010
630499150,15cyclictest26565-21munin-node22:19:430
6310991414,0cyclictest0-21swapper/321:30:233
6310991414,0cyclictest0-21swapper/321:25:583
6310991412,1cyclictest29653-21hddtemp_smartct22:15:133
631099140,14cyclictest0-21swapper/323:21:353
631099140,14cyclictest0-21swapper/323:00:263
631099140,14cyclictest0-21swapper/322:45:323
631099140,14cyclictest0-21swapper/322:06:073
631099140,14cyclictest0-21swapper/321:55:383
631099140,14cyclictest0-21swapper/321:38:193
631099140,14cyclictest0-21swapper/321:00:193
631099140,14cyclictest0-21swapper/320:53:563
631099140,14cyclictest0-21swapper/320:40:473
631099140,14cyclictest0-21swapper/319:47:443
631099140,14cyclictest0-21swapper/319:40:053
631099140,14cyclictest0-21swapper/319:32:333
631099140,14cyclictest0-21swapper/319:22:033
631099140,14cyclictest0-21swapper/319:15:043
631099140,14cyclictest0-21swapper/300:35:193
631099140,14cyclictest0-21swapper/300:15:453
631099140,14cyclictest0-21swapper/300:06:593
631099140,0cyclictest0-21swapper/323:55:183
631099140,0cyclictest0-21swapper/323:15:453
631099140,0cyclictest0-21swapper/323:06:413
631099140,0cyclictest0-21swapper/322:25:253
631099140,0cyclictest0-21swapper/321:20:083
631099140,0cyclictest0-21swapper/300:20:463
6309991414,0cyclictest0-21swapper/223:25:352
630999140,14cyclictest0-21swapper/223:31:122
630999140,14cyclictest0-21swapper/223:22:432
630999140,14cyclictest0-21swapper/223:17:332
630999140,14cyclictest0-21swapper/223:05:402
630999140,14cyclictest0-21swapper/222:05:072
630999140,14cyclictest0-21swapper/222:03:032
630999140,14cyclictest0-21swapper/221:57:142
630999140,14cyclictest0-21swapper/221:30:592
630999140,14cyclictest0-21swapper/221:27:002
630999140,14cyclictest0-21swapper/221:03:172
630999140,14cyclictest0-21swapper/220:50:302
630999140,14cyclictest0-21swapper/220:16:512
630999140,14cyclictest0-21swapper/220:10:202
630999140,14cyclictest0-21swapper/219:41:002
630999140,14cyclictest0-21swapper/219:21:082
630999140,14cyclictest0-21swapper/219:18:382
630999140,14cyclictest0-21swapper/200:20:362
630999140,14cyclictest0-21swapper/200:15:192
630999140,0cyclictest0-21swapper/222:47:312
630999140,0cyclictest0-21swapper/221:40:472
630999140,0cyclictest0-21swapper/221:23:342
630999140,0cyclictest0-21swapper/220:40:142
630999140,0cyclictest0-21swapper/220:35:002
630999140,0cyclictest0-21swapper/220:05:142
630999140,0cyclictest0-21swapper/219:30:132
630999140,0cyclictest0-21swapper/200:05:452
6308991414,0cyclictest28122-21timerandwakeup22:10:211
6308991414,0cyclictest0-21swapper/123:20:101
6308991414,0cyclictest0-21swapper/121:51:051
6308991414,0cyclictest0-21swapper/120:41:171
6308991414,0cyclictest0-21swapper/120:15:081
6308991414,0cyclictest0-21swapper/119:31:271
6308991414,0cyclictest0-21swapper/100:25:171
630899140,14cyclictest0-21swapper/123:45:151
630899140,14cyclictest0-21swapper/123:25:021
630899140,14cyclictest0-21swapper/122:55:261
630899140,14cyclictest0-21swapper/122:21:171
630899140,14cyclictest0-21swapper/121:45:411
630899140,14cyclictest0-21swapper/120:56:031
630899140,14cyclictest0-21swapper/120:35:021
630899140,14cyclictest0-21swapper/120:22:261
630899140,13cyclictest0-21swapper/100:15:141
630899140,0cyclictest0-21swapper/122:50:071
630899140,0cyclictest0-21swapper/122:35:101
630899140,0cyclictest0-21swapper/122:30:441
630899140,0cyclictest0-21swapper/119:55:071
630899140,0cyclictest0-21swapper/119:53:311
6304991414,0cyclictest0-21swapper/023:05:100
6304991414,0cyclictest0-21swapper/021:36:090
6304991414,0cyclictest0-21swapper/020:46:000
6304991414,0cyclictest0-21swapper/020:05:580
6304991414,0cyclictest0-21swapper/019:51:530
6304991412,1cyclictest0-21swapper/023:50:170
630499140,14cyclictest0-21swapper/023:46:390
630499140,14cyclictest0-21swapper/023:40:040
630499140,14cyclictest0-21swapper/023:36:200
630499140,14cyclictest0-21swapper/023:16:110
630499140,14cyclictest0-21swapper/022:45:000
630499140,14cyclictest0-21swapper/022:40:120
630499140,14cyclictest0-21swapper/022:38:000
630499140,14cyclictest0-21swapper/022:30:130
630499140,14cyclictest0-21swapper/022:10:410
630499140,14cyclictest0-21swapper/022:03:090
630499140,14cyclictest0-21swapper/021:15:120
630499140,14cyclictest0-21swapper/021:06:270
630499140,14cyclictest0-21swapper/021:00:250
630499140,14cyclictest0-21swapper/019:42:360
630499140,14cyclictest0-21swapper/019:27:480
630499140,0cyclictest0-21swapper/022:51:250
630499140,0cyclictest0-21swapper/021:50:090
630499140,0cyclictest0-21swapper/021:45:400
630499140,0cyclictest0-21swapper/020:54:280
630499140,0cyclictest0-21swapper/020:44:010
630499140,0cyclictest0-21swapper/020:25:050
630499140,0cyclictest0-21swapper/019:30:140
630499140,0cyclictest0-21swapper/000:05:100
6310991313,0cyclictest0-21swapper/323:30:173
6310991313,0cyclictest0-21swapper/322:20:283
6310991313,0cyclictest0-21swapper/320:30:373
6310991313,0cyclictest0-21swapper/320:26:253
6310991313,0cyclictest0-21swapper/320:10:073
6309991313,0cyclictest0-21swapper/223:55:112
6309991313,0cyclictest0-21swapper/200:35:032
6304991313,0cyclictest0-21swapper/000:10:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional