You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-26 - 05:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5.osadl.org (updated Wed Nov 26, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2941799310,2cyclictest303-21systemd-journal22:25:010
2942199300,29cyclictest28206-21munin-run23:40:012
2941899300,29cyclictest22-21ksoftirqd/123:10:121
2941799300,29cyclictest7845-21cron00:05:010
2941799300,29cyclictest626-21gdbus00:10:010
29425992927,1cyclictest0-21swapper/321:40:013
2942599290,28cyclictest2206-21apt-get19:20:123
2942599290,28cyclictest15369-21apt-get20:55:123
29421992927,1cyclictest19780-21munin-run21:05:002
2942199290,28cyclictest4217-21awk22:50:002
2942199290,28cyclictest26157-21apt-get20:10:162
29418992927,1cyclictest5409-21awk20:35:001
2941899290,28cyclictest0-21swapper/121:05:111
2941799290,28cyclictest8302-21cron21:50:000
2941799290,28cyclictest21846-21apt-get00:35:010
2941799290,28cyclictest0-21swapper/000:10:130
29425992825,2cyclictest3058-21cron23:55:013
29425992825,2cyclictest24471-21cron00:40:013
2942599280,0cyclictest3951-21apt-get21:40:123
2942599280,0cyclictest27040-21apt-get21:20:143
2942599280,0cyclictest17718-21apt-get21:00:133
2942599280,0cyclictest0-21swapper/320:40:123
29421992828,0cyclictest0-21swapper/220:25:122
29421992827,1cyclictest11855-21apt-get23:05:152
2942199280,28cyclictest0-21swapper/220:15:142
2942199280,28cyclictest0-21swapper/200:15:132
2942199280,0cyclictest0-21swapper/219:20:122
29418992827,1cyclictest22-21ksoftirqd/120:35:131
2941899280,28cyclictest0-21swapper/120:05:121
2941899280,1cyclictest22-21ksoftirqd/119:30:131
2941899280,0cyclictest0-21swapper/122:15:111
2941899280,0cyclictest0-21swapper/122:15:111
2941799280,0cyclictest0-21swapper/021:55:120
2941799280,0cyclictest0-21swapper/020:00:140
29425992727,0cyclictest0-21swapper/322:00:123
29425992726,1cyclictest29355-21perf22:35:013
2942599270,14cyclictest0-21swapper/323:21:083
29421992727,0cyclictest0-21swapper/221:35:152
29421992727,0cyclictest0-21swapper/221:05:112
29421992727,0cyclictest0-21swapper/220:05:112
29421992725,1cyclictest17688-21cron22:10:002
2942199270,0cyclictest0-21swapper/220:35:132
2942199270,0cyclictest0-21swapper/200:10:132
29418992727,0cyclictest21846-21apt-get00:35:021
2941899270,27cyclictest0-21swapper/123:20:141
2941899270,0cyclictest29922-21apt-get22:35:121
29417992727,0cyclictest31135-21apt-get20:20:120
29417992727,0cyclictest0-21swapper/020:50:120
29417992727,0cyclictest0-21swapper/019:55:120
29417992727,0cyclictest0-21swapper/019:50:130
2941799270,27cyclictest0-21swapper/022:15:120
2941799270,27cyclictest0-21swapper/022:15:120
2941799270,27cyclictest0-21swapper/019:20:120
2941799270,27cyclictest0-21swapper/019:15:110
29425992626,0cyclictest0-21swapper/323:45:023
29425992624,1cyclictest31437-21munin-run21:30:003
29425992624,1cyclictest0-21swapper/320:50:003
29421992626,0cyclictest0-21swapper/219:40:122
29421992626,0cyclictest0-21swapper/219:20:012
29418992626,0cyclictest0-21swapper/123:35:121
29418992626,0cyclictest0-21swapper/121:35:151
29418992626,0cyclictest0-21swapper/121:25:091
29418992626,0cyclictest0-21swapper/119:40:121
29417992626,0cyclictest0-21swapper/021:20:140
29417992626,0cyclictest0-21swapper/019:35:140
29425992525,0cyclictest0-21swapper/321:55:113
29425992524,1cyclictest5780-21apt-get00:00:143
29421992525,0cyclictest0-21swapper/222:35:132
29421992525,0cyclictest0-21swapper/221:15:122
29421992525,0cyclictest0-21swapper/219:45:132
29421992525,0cyclictest0-21swapper/219:35:142
29418992525,0cyclictest211rcuc/122:55:121
29418992525,0cyclictest0-21swapper/121:30:131
29418992525,0cyclictest0-21swapper/100:30:121
29418992523,1cyclictest2176-21apt-get22:45:111
29418992522,2cyclictest17714-21munin-run22:10:001
29417992525,0cyclictest23828-21apt-get20:05:110
29417992525,0cyclictest0-21swapper/022:30:120
29417992525,0cyclictest0-21swapper/019:30:130
29425992424,0cyclictest16803-21apt-get23:15:133
29425992424,0cyclictest0-21swapper/323:25:133
29425992424,0cyclictest0-21swapper/321:45:133
29421992424,0cyclictest0-21swapper/223:20:142
29421992424,0cyclictest0-21swapper/222:40:142
29421992422,1cyclictest6897-21perf19:30:002
29418992424,0cyclictest0-21swapper/120:50:131
29418992424,0cyclictest0-21swapper/119:35:141
29418992423,0cyclictest0-21swapper/100:00:011
29425992323,0cyclictest14510-21apt-get19:45:123
29418992323,0cyclictest28479-21apt-get20:15:141
29418992322,1cyclictest17456-21apt-get00:25:131
29418992317,4cyclictest26315-21wc22:25:191
29417992323,0cyclictest0-21swapper/022:35:120
29417992323,0cyclictest0-21swapper/021:00:130
29417992323,0cyclictest0-21swapper/020:25:120
29417992323,0cyclictest0-21swapper/019:40:120
29425992222,0cyclictest9174-21apt-get23:00:133
29425992222,0cyclictest32271-21apt-get19:15:103
29425992222,0cyclictest29952-21apt-get19:10:143
29425992222,0cyclictest22642-21apt-get22:20:133
29425992221,1cyclictest4499-21apt-get22:50:103
29425992221,1cyclictest0-21swapper/300:15:013
29418992222,0cyclictest10462-21apt-get00:10:131
29418992222,0cyclictest0-21swapper/121:00:131
29418992222,0cyclictest0-21swapper/119:45:131
29417992222,0cyclictest5999-21apt-get20:35:130
29417992222,0cyclictest17183-21perf00:25:010
29417992222,0cyclictest0-21swapper/021:40:120
29417992222,0cyclictest0-21swapper/021:35:150
29417992222,0cyclictest0-21swapper/019:45:120
29417992221,1cyclictest5569-21cut00:00:010
2942599218,0cyclictest0-21swapper/322:25:083
2942599218,0cyclictest0-21swapper/321:06:473
29425992121,0cyclictest0-21swapper/319:30:133
29421992121,0cyclictest0-21swapper/223:10:122
29418992121,0cyclictest0-21swapper/119:20:121
29417992119,1cyclictest24503-21perf00:40:010
29425992020,0cyclictest9488-21apt-get19:35:143
29425992020,0cyclictest11886-21apt-get19:40:123
29421992020,0cyclictest31106-21apt-get23:45:122
29421992020,0cyclictest0-21swapper/219:55:122
29421992020,0cyclictest0-21swapper/200:05:132
29418992020,0cyclictest0-21swapper/123:05:151
29418992020,0cyclictest0-21swapper/100:05:121
29417992019,1cyclictest19523-21munin-run00:30:010
29425991919,0cyclictest22369-21apt-get21:10:133
29425991919,0cyclictest0-21swapper/320:20:133
29425991919,0cyclictest0-21swapper/300:30:123
29421991919,0cyclictest0-21swapper/221:40:132
29421991918,1cyclictest15372-21perf22:05:002
29421991917,1cyclictest303-21systemd-journal23:35:002
29421991916,2cyclictest0-21swapper/220:45:122
29418991919,0cyclictest1008-21apt-get23:50:111
29418991919,0cyclictest0-21swapper/121:40:131
29418991919,0cyclictest0-21swapper/121:15:121
29418991919,0cyclictest0-21swapper/100:15:131
29418991918,1cyclictest1038-21apt-get20:25:121
2941799193,11cyclictest20160-21munin-node22:48:190
29417991919,0cyclictest0-21swapper/020:45:120
29417991917,1cyclictest0-21swapper/022:15:010
29421991818,0cyclictest0-21swapper/200:20:132
29421991814,4cyclictest0-21swapper/219:50:202
29418991818,0cyclictest0-21swapper/120:20:131
29418991818,0cyclictest0-21swapper/119:25:021
29418991817,1cyclictest22-21ksoftirqd/121:24:491
29417991818,0cyclictest0-21swapper/023:20:140
2942199171,4cyclictest0-21swapper/222:15:122
2942199171,4cyclictest0-21swapper/222:15:122
2941899172,13cyclictest0-21swapper/123:25:131
29418991717,0cyclictest0-21swapper/100:01:021
29418991715,2cyclictest0-21swapper/119:15:191
29417991715,1cyclictest6540-21awk22:55:010
29425991616,0cyclictest0-21swapper/300:20:013
2942599160,15cyclictest9182-21ntp_states00:05:183
29421991616,0cyclictest0-21swapper/220:30:122
29421991616,0cyclictest0-21swapper/219:10:022
29421991614,1cyclictest613-21in:imuxsock22:15:012
2942199160,0cyclictest0-21swapper/220:57:582
29418991616,0cyclictest0-21swapper/120:00:141
29418991614,1cyclictest0-21swapper/123:05:011
2941899160,15cyclictest16176-21ntp_states00:20:181
2941799160,15cyclictest15819-21users23:10:210
29425991513,1cyclictest0-21swapper/322:45:003
2942599150,15cyclictest0-21swapper/323:43:333
2942599150,14cyclictest24873-21ntp_states20:05:173
2942599150,14cyclictest0-21swapper/320:50:213
29421991514,1cyclictest15116-21perf20:55:012
29421991513,1cyclictest23076-21/usr/sbin/munin00:35:172
29421991513,1cyclictest0-21swapper/220:20:012
2942199150,15cyclictest0-21swapper/219:30:292
2942199150,15cyclictest0-21swapper/200:29:402
2942199150,14cyclictest0-21swapper/221:10:212
29418991515,0cyclictest0-21swapper/123:35:001
29418991514,1cyclictest30740-21latency_hist23:45:011
2941899150,1cyclictest0-21swapper/122:00:001
29417991515,0cyclictest6826-21apt-get22:55:120
29417991515,0cyclictest15646-21apt-get22:05:140
29417991515,0cyclictest0-21swapper/000:15:130
2941799150,14cyclictest0-21swapper/023:00:090
29425991414,0cyclictest0-21swapper/322:57:413
29425991414,0cyclictest0-21swapper/321:50:143
29425991414,0cyclictest0-21swapper/320:35:133
29425991414,0cyclictest0-21swapper/320:19:163
29425991414,0cyclictest0-21swapper/320:00:133
2942599140,14cyclictest0-21swapper/323:55:453
2942599140,14cyclictest0-21swapper/323:30:073
2942599140,14cyclictest0-21swapper/323:06:113
2942599140,14cyclictest0-21swapper/322:10:013
2942599140,14cyclictest0-21swapper/320:29:303
2942599140,14cyclictest0-21swapper/320:11:353
2942599140,14cyclictest0-21swapper/319:55:283
2942599140,14cyclictest0-21swapper/319:52:263
2942599140,14cyclictest0-21swapper/319:26:373
2942599140,14cyclictest0-21swapper/300:25:153
2942599140,0cyclictest0-21swapper/323:35:013
2942599140,0cyclictest0-21swapper/323:12:113
2942599140,0cyclictest0-21swapper/322:35:463
2942599140,0cyclictest0-21swapper/320:30:043
2942599140,0cyclictest0-21swapper/300:22:043
29421991414,0cyclictest0-21swapper/222:31:242
29421991414,0cyclictest0-21swapper/222:27:382
29421991414,0cyclictest0-21swapper/221:46:052
29421991413,1cyclictest0-21swapper/223:25:252
29421991412,1cyclictest0-21swapper/221:20:192
2942199140,14cyclictest0-21swapper/223:58:432
2942199140,14cyclictest0-21swapper/223:40:432
2942199140,14cyclictest0-21swapper/223:17:192
2942199140,14cyclictest0-21swapper/222:57:052
2942199140,14cyclictest0-21swapper/221:50:212
2942199140,14cyclictest0-21swapper/221:29:382
2942199140,14cyclictest0-21swapper/200:01:142
2942199140,0cyclictest0-21swapper/223:50:562
2942199140,0cyclictest0-21swapper/223:00:482
2942199140,0cyclictest0-21swapper/222:22:002
2942199140,0cyclictest0-21swapper/220:40:162
2942199140,0cyclictest0-21swapper/220:01:392
2942199140,0cyclictest0-21swapper/200:31:382
29418991414,0cyclictest0-21swapper/122:21:201
29418991414,0cyclictest0-21swapper/121:50:021
29418991414,0cyclictest0-21swapper/121:45:311
29418991414,0cyclictest0-21swapper/121:11:591
29418991414,0cyclictest0-21swapper/120:10:161
29418991414,0cyclictest0-21swapper/119:50:141
29418991412,1cyclictest0-21swapper/120:40:171
2941899140,14cyclictest0-21swapper/123:46:001
2941899140,14cyclictest0-21swapper/122:50:231
2941899140,14cyclictest0-21swapper/122:40:551
2941899140,14cyclictest0-21swapper/122:11:121
2941899140,14cyclictest0-21swapper/120:45:071
2941899140,14cyclictest0-21swapper/119:58:471
2941899140,14cyclictest0-21swapper/119:13:311
2941899140,0cyclictest0-21swapper/123:15:271
2941899140,0cyclictest0-21swapper/122:30:131
2941899140,0cyclictest0-21swapper/122:00:571
2941899140,0cyclictest0-21swapper/120:55:161
29417991414,0cyclictest0-21swapper/023:45:110
29417991414,0cyclictest0-21swapper/023:40:480
29417991414,0cyclictest0-21swapper/022:40:220
29417991414,0cyclictest0-21swapper/021:11:390
29417991414,0cyclictest0-21swapper/019:25:390
29417991412,1cyclictest22523-21ntp_states23:25:190
2941799140,1cyclictest22078-21cron21:10:000
2941799140,14cyclictest0-21swapper/023:16:040
2941799140,14cyclictest0-21swapper/021:25:020
2941799140,14cyclictest0-21swapper/021:15:310
2941799140,14cyclictest0-21swapper/020:40:220
2941799140,14cyclictest0-21swapper/020:30:290
2941799140,14cyclictest0-21swapper/020:15:310
2941799140,14cyclictest0-21swapper/020:12:060
2941799140,14cyclictest0-21swapper/019:11:340
2941799140,0cyclictest0-21swapper/023:51:280
2941799140,0cyclictest0-21swapper/023:31:370
2941799140,0cyclictest0-21swapper/022:00:470
2941799140,0cyclictest0-21swapper/021:54:020
2941799140,0cyclictest0-21swapper/021:31:400
2941799140,0cyclictest0-21swapper/020:56:050
29425991313,0cyclictest0-21swapper/322:46:303
2942599130,0cyclictest0-21swapper/322:15:333
2942599130,0cyclictest0-21swapper/322:15:333
2942599130,0cyclictest0-21swapper/322:05:223
29421991313,0cyclictest31695-21apt-get21:30:132
29421991313,0cyclictest0-21swapper/222:51:122
29421991313,0cyclictest0-21swapper/221:57:412
29417991313,0cyclictest0-21swapper/023:35:070
29417991313,0cyclictest0-21swapper/023:05:450
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional