You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-03 - 05:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5.osadl.org (updated Thu Jul 03, 2025 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3158899300,29cyclictest28872-21cron21:25:003
31578993027,2cyclictest0-21swapper/121:45:131
3157899300,29cyclictest615-21rs:main1
3157899300,29cyclictest26551-21cron21:20:001
31573993027,2cyclictest7287-21sh00:05:000
3157399300,1cyclictest303-21systemd-journal20:20:020
31588992928,1cyclictest331rcuc/322:00:003
3158899290,28cyclictest0-21swapper/323:40:173
31584992927,1cyclictest26004-21awk20:10:002
3158499290,28cyclictest0-21swapper/222:00:142
3157899290,28cyclictest8993-21apt-get21:50:141
3157899290,28cyclictest21593-21apt-get21:10:021
3157899290,28cyclictest0-21swapper/123:05:121
3157899290,28cyclictest0-21swapper/121:40:131
3157899290,1cyclictest30001-21apt-get22:35:181
3157399290,28cyclictest1406-21idleruntime-cro19:15:010
3157399290,28cyclictest13660-21apt-get22:00:130
31588992828,0cyclictest31795-21apt-get19:10:123
3158899280,0cyclictest0-21swapper/320:55:143
31584992827,1cyclictest4573-21apt-get22:50:142
31584992827,1cyclictest11315-21apt-get21:55:132
31584992827,1cyclictest1131-21perf21:35:012
31584992826,1cyclictest9242-21apt-get23:00:152
31584992826,1cyclictest0-21swapper/220:00:022
3158499280,1cyclictest19246-21perf00:30:012
3158499280,0cyclictest25347-21apt-get22:25:132
3158499280,0cyclictest0-21swapper/222:05:132
31578992828,0cyclictest0-21swapper/122:20:151
31578992828,0cyclictest0-21swapper/119:45:031
3157899280,28cyclictest0-21swapper/119:35:141
3157899280,0cyclictest0-21swapper/100:25:121
31573992828,0cyclictest4048-21apt-get19:20:120
31573992827,1cyclictest18604-21apt-get23:20:120
3157399280,0cyclictest0-21swapper/023:50:150
31588992727,0cyclictest0-21swapper/319:35:123
31588992726,1cyclictest32063-21perf22:40:003
31588992726,0cyclictest0-21swapper/322:25:133
3158899270,27cyclictest0-21swapper/321:50:133
31584992727,0cyclictest30550-21apt-get23:45:162
31584992727,0cyclictest0-21swapper/223:00:012
31584992727,0cyclictest0-21swapper/221:20:122
31584992727,0cyclictest0-21swapper/219:25:132
3158499270,0cyclictest27669-21apt-get22:30:112
31578992727,0cyclictest26291-21apt-get20:10:131
31578992727,0cyclictest0-21swapper/121:05:151
31578992727,0cyclictest0-21swapper/119:10:131
31578992727,0cyclictest0-21swapper/100:05:041
31578992726,1cyclictest10184-21apt-get20:45:141
3157899270,0cyclictest0-21swapper/121:25:131
3157899270,0cyclictest0-21swapper/100:35:151
31573992727,0cyclictest0-21swapper/022:30:120
31573992727,0cyclictest0-21swapper/022:15:150
31573992727,0cyclictest0-21swapper/021:05:140
31573992727,0cyclictest0-21swapper/020:20:120
31573992725,1cyclictest609-21dbus-daemon22:50:000
3157399270,27cyclictest0-21swapper/021:35:160
3157399270,27cyclictest0-21swapper/021:20:120
3157399270,0cyclictest0-21swapper/022:20:170
3157399270,0cyclictest0-21swapper/000:15:130
31588992626,0cyclictest18310-21sendmail-msp23:20:003
31588992626,0cyclictest0-21swapper/321:25:143
31588992624,1cyclictest25277-21cron23:35:013
31584992625,1cyclictest26578-21perf21:20:012
31573992626,0cyclictest0-21swapper/020:50:110
31588992525,0cyclictest21646-21apt-get20:00:153
31588992525,0cyclictest0-21swapper/323:30:003
31588992525,0cyclictest0-21swapper/322:40:173
31588992525,0cyclictest0-21swapper/320:10:123
31584992525,0cyclictest0-21swapper/200:15:132
31578992525,0cyclictest415-21apt-get23:50:151
31573992525,0cyclictest0-21swapper/022:50:140
31573992525,0cyclictest0-21swapper/020:35:120
31573992525,0cyclictest0-21swapper/019:40:020
31588992424,0cyclictest15981-21apt-get22:05:153
31588992424,0cyclictest0-21swapper/321:15:143
31588992424,0cyclictest0-21swapper/320:40:133
31588992424,0cyclictest0-21swapper/300:25:133
31584992424,0cyclictest0-21swapper/223:10:152
31584992422,1cyclictest0-21swapper/223:40:012
31578992424,0cyclictest0-21swapper/122:30:121
31588992323,0cyclictest6670-21apt-get21:45:123
31588992321,1cyclictest609-21dbus-daemon23:05:013
31584992323,0cyclictest29167-21apt-get21:25:132
31584992323,0cyclictest0-21swapper/221:35:152
31584992323,0cyclictest0-21swapper/220:35:132
31584992323,0cyclictest0-21swapper/220:20:122
31584992323,0cyclictest0-21swapper/200:10:162
31584992322,1cyclictest1695-21apt-get19:15:102
31578992323,0cyclictest20678-21apt-get22:15:131
31573992323,0cyclictest0-21swapper/000:30:150
31588992222,0cyclictest2256-21apt-get22:45:173
31588992222,0cyclictest0-21swapper/323:45:173
31588992222,0cyclictest0-21swapper/300:10:153
31584992222,0cyclictest0-21swapper/223:55:142
31584992222,0cyclictest0-21swapper/222:10:122
31584992220,1cyclictest9901-21cron00:10:012
31578992222,0cyclictest0-21swapper/121:00:131
31573992222,0cyclictest0-21swapper/022:25:140
31573992222,0cyclictest0-21swapper/022:05:130
31573992222,0cyclictest0-21swapper/000:30:010
31588992121,0cyclictest14863-21apt-get00:20:143
31588992120,1cyclictest6898-21apt-get22:55:163
31588992119,1cyclictest0-21swapper/322:20:013
31584992121,0cyclictest19508-21apt-get00:30:152
31584992121,0cyclictest0-21swapper/220:50:112
31578992121,0cyclictest0-21swapper/122:50:131
31578992120,1cyclictest12256-21perf00:15:021
31578992119,1cyclictest16900-21cron00:25:011
31573992121,0cyclictest0-21swapper/023:55:150
31573992121,0cyclictest0-21swapper/020:05:150
31573992119,1cyclictest1127-21apt-get21:35:010
31578992020,0cyclictest0-21swapper/123:45:161
31578992020,0cyclictest0-21swapper/119:25:131
31573992018,1cyclictest21372-21munin-run20:00:010
31588991919,0cyclictest17173-21apt-get21:00:133
31588991918,1cyclictest3200-21apt-get20:30:163
31584991919,0cyclictest0-21swapper/200:20:132
31578991919,0cyclictest0-21swapper/122:55:151
31573991919,0cyclictest0-21swapper/021:55:120
31573991919,0cyclictest0-21swapper/019:25:130
31588991818,0cyclictest4347-21apt-get21:40:133
31588991818,0cyclictest0-21swapper/322:00:133
31588991818,0cyclictest0-21swapper/319:30:133
31584991818,0cyclictest5258-21perf20:35:012
31584991818,0cyclictest0-21swapper/223:20:132
31578991818,0cyclictest0-21swapper/122:40:171
31578991818,0cyclictest0-21swapper/121:20:111
31573991818,0cyclictest0-21swapper/023:45:160
31573991818,0cyclictest0-21swapper/023:00:160
31573991818,0cyclictest0-21swapper/023:00:010
31588991717,0cyclictest0-21swapper/300:00:143
3158899170,0cyclictest0-21swapper/300:05:173
31584991717,0cyclictest0-21swapper/220:05:002
31584991715,1cyclictest15730-21munin-run19:45:012
31578991717,0cyclictest0-21swapper/120:15:121
3157899170,16cyclictest21465-21cut20:00:011
31573991717,0cyclictest0-21swapper/021:20:000
31588991614,2cyclictest20287-21chrt23:20:413
31588991614,1cyclictest0-21swapper/319:30:003
31588991614,1cyclictest0-21swapper/300:40:023
3158899160,15cyclictest7569-21latency_hist20:40:013
3158499160,15cyclictest24237-21perf21:15:002
31578991616,0cyclictest0-21swapper/122:00:131
31578991616,0cyclictest0-21swapper/120:55:141
31578991614,1cyclictest7299-21apt-get00:05:011
3157899160,2cyclictest3796-21sendmail-msp21:40:001
3157899160,15cyclictest566-21perf20:25:011
31573991616,0cyclictest0-21swapper/023:10:160
31573991614,1cyclictest27961-21munin-run23:40:010
3157399160,15cyclictest24241-21munin-run21:15:000
3157399160,15cyclictest0-21swapper/019:40:020
31588991515,0cyclictest341ksoftirqd/320:00:023
31588991515,0cyclictest0-21swapper/320:15:123
31588991514,1cyclictest0-21swapper/320:10:013
31588991513,1cyclictest2751-21ntp_states19:15:173
31588991513,1cyclictest0-21swapper/321:10:203
3158899150,14cyclictest3835-21ntp_states23:55:203
3158499150,1cyclictest2978-21latency_hist20:30:012
3158499150,14cyclictest29664-21ntp_states20:15:182
3158499150,14cyclictest0-21swapper/200:35:152
31578991515,0cyclictest0-21swapper/123:15:141
31578991513,1cyclictest9210-21/usr/sbin/munin23:00:191
31578991513,1cyclictest25296-21cron23:35:011
31578991513,1cyclictest1142-21munin-run21:35:011
3157899150,1cyclictest7571-21perf20:40:001
3157899150,1cyclictest0-21swapper/122:45:251
3157899150,1cyclictest0-21swapper/122:25:191
3157899150,1cyclictest0-21swapper/120:55:001
3157899150,15cyclictest2927-21perf20:30:011
3157899150,15cyclictest13439-21date22:00:011
31573991515,0cyclictest10199-21apt-get00:10:150
31573991515,0cyclictest0-21swapper/021:25:130
3157399150,1cyclictest22652-21cut20:00:180
3157399150,1cyclictest18591-21switchtime21:00:220
3157399150,15cyclictest9725-21ntpq19:30:200
3157399150,0cyclictest0-21swapper/019:54:070
31588991414,0cyclictest0-21swapper/323:37:253
31588991414,0cyclictest0-21swapper/321:05:033
31588991414,0cyclictest0-21swapper/319:23:103
3158899140,14cyclictest0-21swapper/323:52:163
3158899140,14cyclictest0-21swapper/323:13:043
3158899140,14cyclictest0-21swapper/323:05:133
3158899140,14cyclictest0-21swapper/322:21:323
3158899140,14cyclictest0-21swapper/322:10:513
3158899140,14cyclictest0-21swapper/320:47:313
3158899140,14cyclictest0-21swapper/319:52:343
3158899140,14cyclictest0-21swapper/319:45:443
3158899140,14cyclictest0-21swapper/300:32:363
3158899140,14cyclictest0-21swapper/300:16:543
3158899140,0cyclictest0-21swapper/322:52:583
3158899140,0cyclictest0-21swapper/322:31:053
3158899140,0cyclictest0-21swapper/321:35:083
3158899140,0cyclictest0-21swapper/321:31:453
3158899140,0cyclictest0-21swapper/320:54:453
3158899140,0cyclictest0-21swapper/320:20:073
31584991414,0cyclictest0-21swapper/221:00:022
31584991412,1cyclictest0-21swapper/220:10:152
3158499140,14cyclictest0-21swapper/223:50:052
3158499140,14cyclictest0-21swapper/223:30:532
3158499140,14cyclictest0-21swapper/223:27:012
3158499140,14cyclictest0-21swapper/222:45:192
3158499140,14cyclictest0-21swapper/222:40:582
3158499140,14cyclictest0-21swapper/222:21:442
3158499140,14cyclictest0-21swapper/222:15:382
3158499140,14cyclictest0-21swapper/221:07:082
3158499140,14cyclictest0-21swapper/220:55:392
3158499140,14cyclictest0-21swapper/220:48:192
3158499140,14cyclictest0-21swapper/220:40:282
3158499140,14cyclictest0-21swapper/219:47:262
3158499140,14cyclictest0-21swapper/219:35:262
3158499140,14cyclictest0-21swapper/219:20:352
3158499140,14cyclictest0-21swapper/200:01:162
3158499140,0cyclictest0-21swapper/223:40:502
3158499140,0cyclictest0-21swapper/223:15:232
3158499140,0cyclictest0-21swapper/223:06:252
3158499140,0cyclictest0-21swapper/221:50:462
3158499140,0cyclictest0-21swapper/221:45:142
3158499140,0cyclictest0-21swapper/219:50:212
3158499140,0cyclictest0-21swapper/219:30:282
3158499140,0cyclictest0-21swapper/219:10:142
31578991414,0cyclictest2784-21apt-get23:55:141
31578991414,0cyclictest0-21swapper/122:10:291
31578991414,0cyclictest0-21swapper/122:05:011
31578991414,0cyclictest0-21swapper/120:30:261
31578991414,0cyclictest0-21swapper/119:50:201
31578991413,1cyclictest23949-21acpi20:05:141
3157899140,14cyclictest0-21swapper/123:40:061
3157899140,14cyclictest0-21swapper/123:35:241
3157899140,14cyclictest0-21swapper/123:11:081
3157899140,14cyclictest0-21swapper/119:40:081
3157899140,14cyclictest0-21swapper/119:30:181
3157899140,14cyclictest0-21swapper/119:15:271
3157899140,14cyclictest0-21swapper/100:15:131
3157899140,0cyclictest0-21swapper/123:25:181
3157899140,0cyclictest0-21swapper/123:20:251
3157899140,0cyclictest0-21swapper/120:41:131
3157899140,0cyclictest0-21swapper/120:01:141
3157899140,0cyclictest0-21swapper/119:20:401
31573991414,0cyclictest0-21swapper/023:31:220
31573991414,0cyclictest0-21swapper/021:50:210
31573991414,0cyclictest0-21swapper/021:41:550
31573991414,0cyclictest0-21swapper/020:25:280
31573991414,0cyclictest0-21swapper/019:15:100
31573991414,0cyclictest0-21swapper/000:35:070
3157399140,14cyclictest0-21swapper/023:40:070
3157399140,14cyclictest0-21swapper/023:25:070
3157399140,14cyclictest0-21swapper/023:15:490
3157399140,14cyclictest0-21swapper/023:05:540
3157399140,14cyclictest0-21swapper/022:41:550
3157399140,14cyclictest0-21swapper/022:35:510
3157399140,14cyclictest0-21swapper/021:45:220
3157399140,14cyclictest0-21swapper/020:55:120
3157399140,14cyclictest0-21swapper/020:45:190
3157399140,14cyclictest0-21swapper/020:41:440
3157399140,14cyclictest0-21swapper/020:32:210
3157399140,0cyclictest0-21swapper/022:10:090
3157399140,0cyclictest0-21swapper/020:10:320
3157399140,0cyclictest0-21swapper/019:46:350
3157399140,0cyclictest0-21swapper/000:20:360
3157399140,0cyclictest0-21swapper/000:05:080
31584991313,0cyclictest0-21swapper/222:35:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional