You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 09:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Mon Mar 16, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
319059912766,56cyclictest31883-21cyclictest21:40:460
31924991252,87cyclictest7324-21apt-get22:00:242
106821200,3sleep2371rcuc/223:35:402
31924991113,89cyclictest13189-21df23:56:022
31905991103,23cyclictest352-21runrttasks21:21:000
31924991083,67cyclictest24944-21apt-get00:15:382
31905991086,10cyclictest0-21swapper/019:05:150
3171521070,5sleep10-21swapper/123:32:521
319059910544,55cyclictest4349-21apt-key00:35:140
319059910510,12cyclictest0-21swapper/000:03:010
31905991024,30cyclictest274442apt-key20:30:110
31905991024,30cyclictest274442apt-key20:30:110
319059910241,56cyclictest0-21swapper/023:45:090
31905991023,40cyclictest24141-21/usr/sbin/munin22:26:420
31905991023,36cyclictest11046-21diskstats22:05:290
31905991016,27cyclictest27164-21unixbench_singl22:30:470
31905991016,27cyclictest27164-21unixbench_singl22:30:470
31905991013,27cyclictest352-21runrttasks23:46:490
319059910044,51cyclictest31883-21cyclictest23:07:210
319059910036,58cyclictest0-21swapper/022:25:120
319059910036,58cyclictest0-21swapper/022:25:120
31905991003,24cyclictest3293-21latency_hist23:40:120
31905991003,23cyclictest15337-21ssh23:59:490
3191699992,30cyclictest5950-21seq23:44:461
3190599994,31cyclictest2468-21cat22:45:120
3190599993,29cyclictest352-21runrttasks00:23:250
31905999923,70cyclictest13858-21sh22:10:210
3191699982,30cyclictest29462-21seq20:35:211
3190599984,22cyclictest352-21runrttasks19:56:590
31905999733,57cyclictest0-21swapper/020:20:460
3190599973,32cyclictest4989-21latency_hist21:00:120
3190599972,21cyclictest1570-21latency19:10:200
31916999640,31cyclictest352-21runrttasks21:24:551
3190599964,31cyclictest16263-21grep19:55:130
3190599964,27cyclictest8227-21/usr/sbin/munin19:30:460
3190599964,27cyclictest8227-21/usr/sbin/munin19:30:460
3190599963,26cyclictest352-21runrttasks21:08:470
3190599963,26cyclictest352-21runrttasks00:05:580
3190599963,26cyclictest352-21runrttasks00:05:580
3191699953,31cyclictest23379-21runrttasks21:31:251
3191699952,27cyclictest20731-21seq20:08:211
3191699952,27cyclictest20731-21seq20:08:211
3190599954,84cyclictest12711-21sed23:55:140
3190599954,34cyclictest22774-21/usr/sbin/munin20:15:270
3190599954,33cyclictest20643-21/usr/sbin/munin23:15:420
3190599953,31cyclictest25041-21ssh21:35:180
3190599953,31cyclictest25041-21ssh21:35:180
3190599953,30cyclictest1734-21latency_hist20:50:120
3190599953,30cyclictest1734-21latency_hist20:50:120
31905999516,24cyclictest11625-21rm23:00:170
3177129510,58sleep231796-21latency_hist19:05:142
3191699943,28cyclictest22141-21seq20:10:481
3191699942,26cyclictest7866-21seq19:28:521
3191699942,26cyclictest6343-21seq19:25:001
31905999444,43cyclictest0-21swapper/022:15:470
31905999444,43cyclictest0-21swapper/022:15:470
3190599944,24cyclictest352-21runrttasks20:10:560
3190599943,36cyclictest12992-21sed19:45:120
31905999415,27cyclictest12236-21rm21:13:510
31905999414,28cyclictest24080-21ssh23:20:520
3191699932,28cyclictest14393-21seq19:49:541
3190599934,32cyclictest5859-21fschecks_count22:50:310
3190599934,31cyclictest29438-21/usr/sbin/munin20:35:380
3190599934,29cyclictest24923-21/usr/sbin/munin00:16:130
3190599934,28cyclictest31987-21grep20:40:460
3190599934,28cyclictest31987-21grep20:40:460
3190599933,30cyclictest29663-21seq22:35:370
3190599933,29cyclictest2526-21seq20:50:400
3190599933,28cyclictest8840-21interrupts22:02:100
3190599933,25cyclictest10936-21rm22:59:020
3190599933,25cyclictest10936-21rm22:59:020
3190599932,43cyclictest7179-21rm21:59:480
3190599932,21cyclictest21705-21apt-get00:10:200
3192499924,66cyclictest23108-21ssh22:25:242
31916999248,36cyclictest0-21swapper/121:00:131
3191699922,30cyclictest14087-21seq23:04:301
3191699922,28cyclictest28455-21seq22:34:191
3191699922,28cyclictest28455-21seq22:34:191
3190599924,31cyclictest2629-21/usr/sbin/munin22:45:420
3190599924,31cyclictest2629-21/usr/sbin/munin22:45:420
3190599924,30cyclictest27826-21/usr/sbin/munin20:31:550
3190599924,29cyclictest18542-21sed23:10:440
3190599923,82cyclictest24705-21cron21:35:090
3190599923,33cyclictest32434-21fschecks_count19:06:020
3190599923,30cyclictest1607-21find21:50:210
31905999227,57cyclictest0-21swapper/021:45:260
3190599922,30cyclictest18187-21grep20:00:310
31905999213,27cyclictest20911-21rm21:27:540
31905999213,27cyclictest20911-21rm21:27:540
306542928,57sleep130656-21irqstats19:00:401
31924999118,29cyclictest352-21runrttasks22:47:442
31924999118,29cyclictest352-21runrttasks22:47:442
3190599919,73cyclictest0-21swapper/019:15:190
3190599918,10cyclictest0-21swapper/021:15:310
3190599918,10cyclictest0-21swapper/021:15:310
3190599913,33cyclictest29863-21/usr/sbin/munin23:30:390
31905999132,51cyclictest0-21swapper/023:25:370
3190599903,30cyclictest9859-21/usr/sbin/munin19:35:330
3190599903,26cyclictest20873-21modprobe20:10:090
3190599903,26cyclictest20873-21modprobe20:10:090
3190599894,32cyclictest1065-21latency_hist00:30:110
3190599893,25cyclictest13791-21sshd19:45:440
3190599893,23cyclictest5190-21sort21:00:250
3190599893,20cyclictest5029-21apt-get19:20:210
3190599892,22cyclictest352-21runrttasks19:25:490
3192499882,52cyclictest10035-21/usr/sbin/munin23:51:232
3191699883,7cyclictest0-21swapper/122:10:201
3191699882,31cyclictest9359-21runrttasks21:09:121
3192499873,56cyclictest23884-21apt-get23:20:342
3191699872,8cyclictest0-21swapper/122:45:251
3191699872,8cyclictest0-21swapper/122:45:251
31916998646,33cyclictest6486-21diskmemload21:11:071
3191699863,30cyclictest3063-21runrttasks19:14:271
3191699862,42cyclictest25773-21latency_hist22:30:121
31916998547,31cyclictest0-21swapper/121:35:431
31916998547,31cyclictest0-21swapper/121:35:431
31916998540,37cyclictest18021-21ls20:00:261
3191699853,8cyclictest0-21swapper/119:50:221
3191699853,7cyclictest0-21swapper/100:25:211
3191699852,58cyclictest28061-21apt-get21:40:221
3191699852,41cyclictest6486-21diskmemload22:40:201
31924998417,24cyclictest21658-21ssh22:22:432
31924998417,24cyclictest21658-21ssh22:22:432
3191699843,70cyclictest16889-21sh22:15:231
3191699843,70cyclictest16889-21sh22:15:231
3191699842,40cyclictest830-21/usr/sbin/munin23:35:331
31924998355,20cyclictest27752-21timerandwakeup23:25:452
31916998341,35cyclictest0-21swapper/119:30:461
31916998341,35cyclictest0-21swapper/119:30:461
3191699833,40cyclictest20413-21/usr/sbin/munin22:20:371
3191699833,40cyclictest20413-21/usr/sbin/munin22:20:371
3191699833,24cyclictest23554-21/usr/sbin/munin20:15:451
3191699832,39cyclictest10632-21sh23:51:551
3191699832,23cyclictest9877-21apt-get19:35:201
3191699823,63cyclictest32679-21apt-get20:45:251
3191699823,63cyclictest32679-21apt-get20:45:251
31916998235,41cyclictest3091-21sh00:32:321
3191699823,37cyclictest6438-21ssh23:45:231
3191699823,37cyclictest2478-21/usr/sbin/munin21:50:461
3191699822,37cyclictest31073-21seq21:45:231
31916998158,16cyclictest13253-21grep23:56:081
3191699813,7cyclictest0-21swapper/120:50:201
3191699813,5cyclictest0-21swapper/120:40:261
3191699813,5cyclictest0-21swapper/120:40:261
3191699813,31cyclictest15793-21ssh00:00:221
3191699813,24cyclictest3424-21apt-get19:15:241
3191699812,7cyclictest0-21swapper/100:05:211
3191699812,7cyclictest0-21swapper/100:05:211
3191699812,35cyclictest352-21runrttasks21:00:261
3191699812,35cyclictest26920-21cpu23:25:251
3191699812,34cyclictest25402-21/usr/sbin/munin00:16:111
3191699812,31cyclictest11470-21/usr/sbin/munin19:40:201
31924998056,15cyclictest29447-21kworker/u17:1-ext4-rsv-conversion22:10:122
3191699803,9cyclictest27922-21ssh00:20:231
3191699803,37cyclictest26566-21/usr/sbin/munin20:25:421
3191699803,37cyclictest26566-21/usr/sbin/munin20:25:421
3191699803,37cyclictest22628-21timerwakeupswit00:10:451
3191699803,35cyclictest17314-21latency_hist23:10:111
31924997926,40cyclictest29124-21ssh22:35:212
3191699793,37cyclictest17678-21/usr/sbin/munin20:00:111
3191699793,29cyclictest13458-21/usr/sbin/munin21:15:331
3191699793,29cyclictest13458-21/usr/sbin/munin21:15:331
3191699792,32cyclictest21738-21cron21:30:091
3191699792,32cyclictest21738-21cron21:30:091
3191699792,30cyclictest4613-21ssh21:55:211
31924997818,36cyclictest0-21swapper/200:09:462
31924997818,36cyclictest0-21swapper/200:09:462
31916997848,8cyclictest181rcu_preempt20:31:501
3191699783,7cyclictest0-21swapper/120:20:251
3191699783,35cyclictest23884-21apt-get23:20:261
3191699783,30cyclictest18387-21/usr/sbin/munin23:10:421
3191699782,29cyclictest5792-21cat22:50:271
31924997758,13cyclictest3405-21/usr/sbin/munin19:15:442
31924997710,37cyclictest352-21runrttasks00:25:072
3191699773,40cyclictest32250-21runrttasks19:05:561
3191699773,33cyclictest6486-21diskmemload22:37:521
3191699773,32cyclictest8917-21fschecks_count22:55:301
3191699773,32cyclictest8917-21fschecks_count22:55:301
3191699772,38cyclictest10855-21apt-get22:05:251
3191699772,30cyclictest9236-21/usr/sbin/munin22:02:211
3191699772,28cyclictest22017-21sh23:16:281
3193999763,47cyclictest0-21swapper/323:55:493
31924997652,18cyclictest352-21runrttasks20:55:382
31924997621,23cyclictest19767-21seq23:13:382
31924997616,25cyclictest3409-21seq00:33:262
31924997553,15cyclictest6797-21latency_hist21:05:132
31924997441,22cyclictest6414-21apt-get23:45:242
3192499743,39cyclictest26832-21timerwakeupswit20:25:462
3192499743,39cyclictest26832-21timerwakeupswit20:25:462
3192499742,38cyclictest16251-21ssh21:20:302
31924997412,25cyclictest0-21swapper/223:44:302
31924997346,19cyclictest12631-21timerwakeupswit23:00:472
31924997344,18cyclictest19072-21cpu21:25:252
31924997344,18cyclictest19072-21cpu21:25:252
3192499733,38cyclictest19733-21latency_hist22:20:132
3192499733,38cyclictest19733-21latency_hist22:20:132
31924997317,27cyclictest14327-21seq21:16:132
31924997317,27cyclictest14327-21seq21:16:132
3192499723,31cyclictest6486-21diskmemload22:56:072
3192499723,31cyclictest6486-21diskmemload22:56:072
31924997231,27cyclictest0-21swapper/221:05:342
3192499722,34cyclictest30813-21modprobe20:40:102
31924997151,14cyclictest11950-21netstat19:40:362
3192499713,41cyclictest28138-21fschecks_time20:31:382
31924997122,17cyclictest16657-21seq23:08:192
31924997055,11cyclictest31883-21cyclictest22:32:422
31924997055,11cyclictest31883-21cyclictest22:32:422
31939996910,40cyclictest0-21swapper/323:50:243
31939996812,39cyclictest0-21swapper/300:15:593
31924996851,13cyclictest8238-21kworker/u18:2+events_unbound22:55:092
31924996849,14cyclictest24801-21latency_hist00:15:132
31924996848,15cyclictest27866-21kworker/u17:3+events_unbound21:15:092
31924996833,26cyclictest14692-21apt-get19:50:222
31924996744,12cyclictest0-21swapper/200:25:312
3192499673,43cyclictest6486-21diskmemload21:35:222
3192499673,43cyclictest6486-21diskmemload21:35:222
3192499672,37cyclictest17675-21latency_hist20:00:102
31924996719,36cyclictest31324-21grep20:40:322
31924996719,36cyclictest31324-21grep20:40:322
31924996645,14cyclictest6371-21latency_hist19:25:132
3192499653,34cyclictest22855-21ls20:15:272
3192499653,30cyclictest200-21systemd-journal22:40:532
40562640,4sleep20-21swapper/221:54:332
3193999643,44cyclictest0-21swapper/321:05:283
3193999643,43cyclictest917-21kworker/3:1+events23:45:193
31939996423,25cyclictest0-21swapper/319:05:273
31939996419,28cyclictest0-21swapper/320:30:383
31939996411,40cyclictest0-21swapper/323:20:563
3192499643,38cyclictest2058-21fschecks_time19:10:372
3192499643,28cyclictest32304-21ls19:05:572
31924996429,23cyclictest13846-21apt-get22:10:202
31924996343,14cyclictest29915-21cpu23:30:272
31924996337,18cyclictest0-21swapper/221:55:282
3192499633,42cyclictest22295-21fschecks_count21:30:312
31924996326,27cyclictest20663-21apt-get23:15:202
31924996236,19cyclictest31052-21apt-get21:45:222
31924996236,19cyclictest19444-21latency_hist20:05:122
3192499623,39cyclictest352-21runrttasks20:47:442
3192499623,39cyclictest352-21runrttasks20:47:442
31924996228,23cyclictest13091-21apt-get19:45:262
3192499622,39cyclictest28174-21cpuspeed21:40:262
3193999615,40cyclictest1285-21taskset00:30:123
31924996144,10cyclictest11423-21latency_hist19:40:132
31924996136,16cyclictest8245-21apt-get19:30:242
31924996136,16cyclictest8245-21apt-get19:30:242
3192499613,32cyclictest20883-21latency_hist20:10:112
3192499613,32cyclictest20883-21latency_hist20:10:112
31924996129,23cyclictest16053-21fschecks_time00:00:312
3192499612,39cyclictest352-21runrttasks20:12:362
3192499612,39cyclictest2811-21grep20:50:442
31939996046,8cyclictest0-21swapper/322:25:263
31939996017,36cyclictest0-21swapper/322:00:363
31924996041,15cyclictest6590-21kworker/u19:1+events_unbound19:30:102
3192499602,29cyclictest24499-21cstates20:20:272
3193999583,38cyclictest8147-21chrt19:30:133
3106625820,24sleep30-21swapper/319:00:553
3193999564,37cyclictest16711-21chrt21:20:423
3193999563,43cyclictest23697-21chrt23:20:133
3193999563,37cyclictest5582-21chrt21:00:363
3193999559,35cyclictest30082-21chrt00:23:523
3193999554,32cyclictest30980-21taskset22:37:553
3193999553,38cyclictest12121-21chrt19:40:423
3193999552,41cyclictest26436-21taskset21:36:313
3193999552,41cyclictest26436-21taskset21:36:313
3193999543,37cyclictest1016-21taskset23:35:373
3193999543,36cyclictest3770-21chrt20:55:293
3193999543,34cyclictest17509-21taskset23:10:123
31939995429,14cyclictest217812chrt20:10:433
3193999542,41cyclictest31145-21taskset20:40:273
3193999542,41cyclictest31145-21taskset20:40:273
3193999542,36cyclictest30104-21chrt20:35:413
3193999534,35cyclictest16855-21chrt00:00:463
3193999533,38cyclictest22046-21taskset22:23:483
3193999533,38cyclictest22046-21taskset22:23:483
3193999533,37cyclictest21677-21taskset00:10:183
3193999533,36cyclictest12427-21chrt23:00:443
3193999533,36cyclictest12147-21taskset22:06:183
3193999533,28cyclictest28118-21chrt23:26:093
3193999524,35cyclictest15551-21chrt21:19:413
3193999524,35cyclictest15551-21chrt21:19:413
3193999523,37cyclictest2238-21chrt22:44:563
3193999523,35cyclictest8565-21taskset19:30:343
3193999523,35cyclictest8565-21taskset19:30:343
3193999523,35cyclictest19624-21chrt22:20:123
3193999523,35cyclictest19624-21chrt22:20:123
3193999523,35cyclictest14681-21chrt22:10:443
3193999522,41cyclictest775-21taskset20:45:413
3193999522,41cyclictest775-21taskset20:45:413
3193999522,39cyclictest3216-21taskset20:55:093
3193999514,34cyclictest5288-21chrt21:55:443
3193999513,37cyclictest4143-21chrt00:35:113
3193999513,37cyclictest30272-21chrt23:30:353
3193999513,37cyclictest20559-21chrt00:07:513
3193999513,37cyclictest20559-21chrt00:07:513
3193999513,35cyclictest24492-21chrt21:34:323
3193999513,32cyclictest25935-21chrt20:25:133
3193999504,32cyclictest14448-21chrt19:50:103
3193999503,35cyclictest19786-21chrt20:05:333
3193999503,35cyclictest19786-21chrt20:05:333
3193999503,35cyclictest1902-21taskset21:50:333
3193999503,32cyclictest5195-21chrt23:42:383
3193999502,36cyclictest17688-21chrt20:00:103
3193999493,37cyclictest917-21kworker/3:1+events23:12:163
3193999493,34cyclictest9674-21chrt22:55:463
3193999493,34cyclictest9674-21chrt22:55:463
3193999493,34cyclictest20669-21chrt21:27:153
3193999493,34cyclictest20669-21chrt21:27:153
3193999493,33cyclictest7265-21chrt22:52:133
3193999493,33cyclictest28403-21chrt21:40:323
3193999493,33cyclictest27441-21chrt20:30:113
3193999493,33cyclictest27441-21chrt20:30:113
3193999493,30cyclictest4985-21chrt22:49:133
3193999493,30cyclictest4985-21chrt22:49:133
3193999492,38cyclictest2036-21chrt19:10:373
3193999492,35cyclictest12564-21chrt21:14:463
3193999483,33cyclictest14939-21chrt19:50:343
3193999483,32cyclictest26723-21chrt22:30:423
3193999483,32cyclictest26723-21chrt22:30:423
3193999482,29cyclictest31197-21chrt21:45:273
3193999473,31cyclictest18309-21chrt20:00:353
31939994715,19cyclictest0-21swapper/319:35:263
3193999444,28cyclictest3406-21chrt19:15:193
3193999433,26cyclictest5897-21chrt19:20:453
3193999393,26cyclictest22775-21taskset20:15:213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional