You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 21:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Tue May 19, 2026 12:44:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27528215812,121sleep127567-21/usr/sbin/munin06:56:201
28716991336,97cyclictest3625-21apt-get09:01:130
28716991336,97cyclictest3625-21apt-get09:01:130
28716991329,105cyclictest23353-21apt-get11:21:120
28716991163,24cyclictest5926-21open_inodes10:51:250
28716991162,23cyclictest24648-21seq09:35:000
28716991143,28cyclictest6508-21sh09:58:230
287169911427,56cyclictest0-21swapper/009:36:270
28716991093,26cyclictest10424-21sh11:53:030
28716991093,26cyclictest10424-21sh11:53:030
28716991093,23cyclictest17759-21ssh09:22:230
28716991093,21cyclictest26109-21ssh12:20:170
28716991092,30cyclictest4933-21latency_hist10:50:550
28716991082,29cyclictest18319-21ssh11:11:320
28716991052,23cyclictest27232-21sed12:21:210
28716991043,24cyclictest28717-21cat07:00:550
28716991042,22cyclictest24161-21rm10:27:500
28716991042,22cyclictest24161-21rm10:27:500
28716991032,24cyclictest23133-21ssh12:15:230
28716991032,24cyclictest23133-21ssh12:15:230
28730991022,42cyclictest14334-21taskset10:11:232
287169910246,50cyclictest1791-21ssh09:51:070
28716991023,25cyclictest32109-21timerandwakeup09:46:280
28716991023,25cyclictest32109-21timerandwakeup09:46:280
28716991023,22cyclictest3491-21seq08:59:210
28716991023,22cyclictest3491-21seq08:59:210
28716991022,35cyclictest28270-21runrttasks09:41:060
28716991013,25cyclictest30979-21ssh11:33:010
28716991013,25cyclictest12553-21ssh11:02:080
28716991013,23cyclictest27390-21diskmemload10:32:510
28716991012,23cyclictest31424-21ssh12:28:290
28716991003,25cyclictest9273-21ssh10:02:490
28716991003,25cyclictest9273-21ssh10:02:490
28716991003,24cyclictest22020-21ssh10:25:120
28716991003,23cyclictest12555-21wc11:56:250
28721999947,8cyclictest0-21swapper/107:31:501
28721999931,24cyclictest1-21systemd12:30:511
2871699993,21cyclictest6900-21/usr/sbin/munin11:46:330
2871699992,73cyclictest17241-21ntp_offset10:16:200
2871699992,27cyclictest12127-21fschecks_time07:46:140
2871699992,25cyclictest2374-21systemd-journal09:17:150
2871699992,24cyclictest21995-21taskset11:18:330
2871699992,24cyclictest21995-21taskset11:18:330
2871699992,22cyclictest17085-21latency_hist11:10:550
28730999846,8cyclictest0-21swapper/210:21:262
2871699983,22cyclictest12689-21ssh09:15:320
2871699982,23cyclictest184982/usr/sbin/munin08:06:130
2871699973,21cyclictest16452-21cron08:00:510
2871699972,22cyclictest26456-21cstates11:26:080
28730999654,32cyclictest6646-21kworker/u19:0+nfsiod12:03:152
2873099962,41cyclictest3818-21grep11:41:262
2871699963,30cyclictest6321-21sendmail-msp07:30:520
2871699963,30cyclictest25049-21grep08:26:140
2871699963,24cyclictest31656-21latency_hist10:40:540
2871699963,22cyclictest9779-21ssh10:57:420
2871699962,21cyclictest13658-21/usr/sbin/munin10:11:010
28716999612,21cyclictest9314-21sh09:09:260
2871699953,29cyclictest11637-21unin-run07:45:520
2871699953,29cyclictest11637-21unin-run07:45:520
2871699952,28cyclictest942-21fschecks_time08:51:360
28730999444,9cyclictest0-21swapper/212:26:262
28721999439,7cyclictest0-21swapper/107:21:251
28721999436,9cyclictest0-21swapper/111:20:531
28721999436,9cyclictest0-21swapper/111:20:531
2871699943,26cyclictest328-21grep07:11:140
2871699943,21cyclictest4089-21ssh11:41:590
2871699942,23cyclictest19607-21ssh12:08:500
2740329414,45sleep227221-21/usr/sbin/munin06:56:172
28721999373,14cyclictest28704-21cyclictest07:46:171
28721999341,7cyclictest0-21swapper/108:26:231
2871699934,10cyclictest0-21swapper/008:40:550
2871699933,28cyclictest31232-21/usr/sbin/munin08:46:270
2871699933,26cyclictest18080-21idleruntime-cro08:05:510
2871699933,23cyclictest32149-21/usr/sbin/munin10:41:160
2871699933,20cyclictest26780-21/usr/sbin/munin08:31:170
2871699932,24cyclictest32246-21ssh11:35:560
2871699932,24cyclictest32246-21ssh11:35:560
2871699932,21cyclictest13480-21taskset07:50:590
28721999276,9cyclictest14110-21kworker/u17:1-xprtiod09:17:271
28721999239,8cyclictest0-21swapper/112:21:241
28721999238,8cyclictest0-21swapper/107:01:261
28721999237,8cyclictest0-21swapper/111:06:151
2871699923,30cyclictest352-21runrttasks07:01:230
2871699923,29cyclictest2679-21grep07:16:290
2872199917,37cyclictest22302-21ssh09:31:011
28721999135,8cyclictest0-21swapper/110:30:561
28721999135,8cyclictest0-21swapper/110:30:561
2871699913,29cyclictest19650-21/usr/sbin/munin09:26:140
2871699913,29cyclictest14714-21ssh12:01:040
2871699913,28cyclictest8384-21latency_hist07:35:540
2871699913,21cyclictest10920-21fschecks_count10:06:130
2871699912,31cyclictest29999-21uname08:41:160
2871699912,21cyclictest21270-21taskset08:15:290
2873099902,34cyclictest23800-21find08:21:232
28721999041,13cyclictest16314-21/usr/sbin/munin09:21:271
28721999027,14cyclictest0-21swapper/107:37:451
28721999027,14cyclictest0-21swapper/107:37:451
2871699903,25cyclictest22187-21/usr/sbin/munin08:16:260
2871699902,30cyclictest3706-21cut07:21:160
2871699902,27cyclictest352-21runrttasks07:05:590
28721998967,16cyclictest3451-21ssh11:41:221
28730998867,15cyclictest3630-21diskmemload11:51:162
28730998867,15cyclictest3630-21diskmemload11:51:162
28721998840,11cyclictest0-21swapper/109:56:421
2872199883,35cyclictest29624-21/usr/sbin/munin08:41:261
2871699883,26cyclictest23268-21cpuspeed08:21:070
2871699882,35cyclictest8641-21/usr/sbin/munin07:36:020
2871699882,35cyclictest8641-21/usr/sbin/munin07:36:020
28730998765,11cyclictest0-21swapper/209:31:282
2873099872,40cyclictest7220-21kernelversion07:31:432
2872199873,34cyclictest29119-21timerandwakeup09:41:261
2873099862,32cyclictest352-21runrttasks09:20:042
28730998566,10cyclictest0-21swapper/208:41:132
28730998564,10cyclictest0-21swapper/211:26:092
2872199853,37cyclictest20325-21latency_hist12:10:531
28721998467,11cyclictest6879-21unixbench_singl11:46:311
28721998436,11cyclictest3630-21diskmemload12:11:541
28721998436,11cyclictest3630-21diskmemload12:11:541
28730998364,13cyclictest26969-21fschecks_time12:21:152
28721998367,10cyclictest11388-21ssh10:06:251
2872199834,36cyclictest21313-21latency_hist08:15:531
28721998337,11cyclictest3630-21diskmemload11:28:471
28735998210,43cyclictest0-21swapper/311:21:143
28730998266,9cyclictest0-21swapper/211:01:202
28721998266,10cyclictest24196-21latency11:22:141
28721998159,16cyclictest28704-21cyclictest07:51:241
2872199813,35cyclictest32657-21if_err_eth011:36:141
2872199813,35cyclictest32657-21if_err_eth011:36:141
2872199813,34cyclictest10707-21ntp_kernel_err09:11:181
2872199813,29cyclictest2252-21/usr/sbin/munin10:46:041
28730998057,9cyclictest181rcu_preempt09:05:552
28730998057,9cyclictest181rcu_preempt09:05:552
28730998051,8cyclictest181rcu_preempt08:56:172
28730998051,8cyclictest181rcu_preempt08:56:172
28730998041,10cyclictest17503-21ssh09:21:392
28721998033,14cyclictest352-21runrttasks11:59:131
28730997861,13cyclictest28704-21cyclictest10:06:282
28730997854,18cyclictest30776-21kworker/u19:1+nfsiod10:16:312
28721997862,8cyclictest0-21swapper/109:06:241
28721997854,18cyclictest15257-21sh12:01:201
28730997760,9cyclictest0-21swapper/211:11:252
28721997748,21cyclictest23866-21fschecks_count12:16:121
28721997735,9cyclictest181rcu_preempt10:11:191
28730997657,9cyclictest0-21swapper/211:46:162
28730997559,10cyclictest8634-21/usr/sbin/munin07:37:552
28730997559,10cyclictest8634-21/usr/sbin/munin07:37:552
28730997557,9cyclictest0-21swapper/208:11:132
28721997556,14cyclictest2053-21latency_hist08:55:531
28721997556,13cyclictest32176-21latency_hist11:35:541
28730997460,9cyclictest28704-21cyclictest10:29:312
28730997460,9cyclictest28704-21cyclictest10:29:312
28730997458,10cyclictest27977-21latency_hist08:35:552
28730997456,8cyclictest0-21swapper/211:16:262
28730997456,8cyclictest0-21swapper/211:16:262
28721997454,14cyclictest25258-21/usr/sbin/munin09:36:231
28721997453,13cyclictest25892-21sh10:31:151
28730997327,21cyclictest0-21swapper/212:10:532
28721997360,7cyclictest0-21swapper/108:31:251
28721997354,9cyclictest0-21swapper/107:06:281
28721997351,16cyclictest300-21rs:main1
28721997347,18cyclictest11085-21ntpd07:26:081
28730997256,7cyclictest0-21swapper/210:32:162
28730997252,13cyclictest503-21latency_hist08:50:552
28730997249,14cyclictest19027-21switchtime08:06:272
28730997242,18cyclictest11056-21kworker/u19:0+nfsiod10:54:232
28721997259,7cyclictest0-21swapper/107:56:151
28721997251,15cyclictest28704-21cyclictest07:41:201
28721997251,15cyclictest28704-21cyclictest07:41:201
28721997250,16cyclictest1-21systemd10:23:151
28730997157,8cyclictest0-21swapper/209:36:282
28730997155,8cyclictest0-21swapper/210:01:142
28730997155,8cyclictest0-21swapper/210:01:142
28730997138,25cyclictest25176-21cat08:26:172
28721997156,9cyclictest352-21runrttasks10:56:221
28721997153,12cyclictest6989-21latency_hist09:05:541
28721997153,12cyclictest6989-21latency_hist09:05:541
28730997054,11cyclictest2374-21systemd-journal08:18:162
28730997054,10cyclictest12877-21systemd-run09:15:532
28721997055,8cyclictest0-21swapper/107:11:261
28721997051,9cyclictest0-21swapper/111:11:171
28730996954,10cyclictest29432-21kworker/u18:0-nfsiod11:59:362
28730996953,8cyclictest0-21swapper/210:46:092
28730996950,12cyclictest297-21systemd-logind12:20:522
28730996948,15cyclictest300-21rs:main2
2872199696,11cyclictest31230-21apt-get09:46:051
2872199696,11cyclictest31230-21apt-get09:46:051
28721996955,9cyclictest28704-21cyclictest09:52:211
28721996954,8cyclictest0-21swapper/108:21:261
28721996953,9cyclictest0-21swapper/110:36:161
28721996929,8cyclictest181rcu_preempt10:51:281
2873099683,43cyclictest7141-21sh10:00:122
28721996848,10cyclictest0-21swapper/110:41:151
28721996843,18cyclictest29432-21kworker/u18:0+nfsiod11:55:231
28721996843,18cyclictest29432-21kworker/u18:0+nfsiod11:55:231
28730996752,8cyclictest0-21swapper/207:30:552
28730996751,11cyclictest2374-21systemd-journal09:44:152
28721996743,18cyclictest3758-21apt-key09:00:541
28721996743,18cyclictest3758-21apt-key09:00:541
28730996651,9cyclictest29432-21kworker/u18:0-rpciod11:36:372
28730996651,9cyclictest29432-21kworker/u18:0-rpciod11:36:372
28730996650,9cyclictest0-21swapper/209:26:282
28730996646,10cyclictest0-21swapper/210:56:142
28730996551,10cyclictest28704-21cyclictest07:43:552
28730996551,10cyclictest28704-21cyclictest07:43:552
28730996549,10cyclictest14598-21sh11:06:162
28721996549,7cyclictest0-21swapper/108:06:291
28721996544,13cyclictest0-21swapper/107:16:131
28721996543,16cyclictest16680-21rm10:16:071
28730996450,10cyclictest28704-21cyclictest10:43:302
28730996448,11cyclictest1140-21memory08:51:402
28721996448,9cyclictest0-21swapper/109:26:231
28721996447,8cyclictest0-21swapper/108:16:301
28721996444,9cyclictest0-21swapper/111:01:171
2873599636,33cyclictest0-21swapper/309:16:343
28730996347,7cyclictest0-21swapper/207:06:162
28730996345,11cyclictest2767-21ssh09:51:282
28730996344,13cyclictest7453-21fschecks_time09:06:142
28730996343,14cyclictest2065-21interrupts07:16:162
28730996331,22cyclictest4666-21modprobe07:25:512
28730996247,8cyclictest0-21swapper/212:15:522
28730996247,8cyclictest0-21swapper/212:15:522
28730996245,12cyclictest16531-21latency_hist08:00:542
28721996249,9cyclictest28704-21cyclictest08:36:121
28721996247,8cyclictest0-21swapper/108:50:531
28730996146,10cyclictest31230-21apt-get09:46:082
28730996146,10cyclictest31230-21apt-get09:46:082
28730996145,8cyclictest0-21swapper/211:31:152
28730996144,11cyclictest29390-21ntp_kernel_pll_07:01:192
2873599604,42cyclictest13492-21chrt10:10:543
28735996038,14cyclictest451rcuc/311:47:443
2873599603,41cyclictest31992-21taskset09:46:263
2873599603,41cyclictest31992-21taskset09:46:263
2873599603,32cyclictest0-21swapper/307:36:063
2873599603,32cyclictest0-21swapper/307:36:063
2873599602,27cyclictest697-21chrt08:51:263
28730996045,9cyclictest18101-21systemd08:05:532
28730996030,22cyclictest13548-21grep07:51:082
28735995940,14cyclictest451rcuc/308:05:423
2873599592,39cyclictest0-21swapper/309:01:083
2873599592,39cyclictest0-21swapper/309:01:083
2873099595,23cyclictest12128-21fschecks_time07:46:142
28721995944,9cyclictest10484-21wc10:05:541
28721995944,9cyclictest10484-21wc10:05:541
28735995839,14cyclictest451rcuc/311:28:073
28735995836,14cyclictest451rcuc/311:11:373
28735995822,29cyclictest0-21swapper/310:31:593
2873099583,44cyclictest366-21/usr/sbin/munin07:11:142
2873099583,36cyclictest29142-21cat10:36:152
2789925812,23sleep30-21swapper/306:56:343
2873599573,41cyclictest6529-21chrt07:30:543
2873099572,41cyclictest28007-21/usr/sbin/munin08:36:072
28735995638,13cyclictest451rcuc/307:15:323
2873599563,39cyclictest26317-21chrt12:20:503
2873599554,38cyclictest20642-21taskset08:11:253
2873599554,36cyclictest11592-21chrt11:55:543
2873599554,36cyclictest11592-21chrt11:55:543
28735995538,11cyclictest451rcuc/308:44:183
28735995534,13cyclictest6252chrt11:36:253
28735995534,13cyclictest6252chrt11:36:253
2873599553,38cyclictest21004-21chrt09:28:103
28735995440,9cyclictest451rcuc/312:28:083
2873599543,39cyclictest17365-21chrt12:05:513
2873599543,38cyclictest18238-21taskset12:06:193
2873599543,36cyclictest25726-21chrt09:36:173
28735995433,14cyclictest451rcuc/308:32:523
28735995412,30cyclictest0-21swapper/307:31:313
2873599534,35cyclictest8610-21chrt10:56:173
2873599534,35cyclictest6367-21taskset10:51:303
28735995339,9cyclictest451rcuc/310:44:263
28735995338,10cyclictest451rcuc/310:18:123
28735995336,9cyclictest451rcuc/311:19:563
28735995336,9cyclictest451rcuc/311:19:563
28735995335,13cyclictest451rcuc/312:00:173
2873599533,38cyclictest10601-21chrt07:41:153
2873599533,38cyclictest10601-21chrt07:41:153
2873599533,34cyclictest3185-21chrt07:20:543
2873599532,40cyclictest5583-21chrt11:45:533
2873599532,38cyclictest4178-21taskset10:48:473
28735995237,10cyclictest451rcuc/308:10:243
28735995236,9cyclictest451rcuc/309:52:133
28735995236,11cyclictest451rcuc/307:10:513
2873599523,37cyclictest9866-21taskset09:10:513
2873599523,35cyclictest24981-21chrt10:30:103
2873599523,35cyclictest24981-21chrt10:30:103
2873599522,37cyclictest28840-21taskset09:41:193
28735995136,10cyclictest451rcuc/307:53:313
28735995135,11cyclictest451rcuc/309:24:103
2873599513,35cyclictest5433-21chrt09:56:233
28735995132,12cyclictest451rcuc/308:18:433
2873599512,35cyclictest25120-21taskset08:26:153
2873599504,33cyclictest14332-21taskset11:06:093
28735995035,8cyclictest451rcuc/308:50:493
28735995034,11cyclictest451rcuc/307:49:393
2873599503,33cyclictest27484-21chrt12:21:273
2873599502,35cyclictest21536-21chrt12:11:263
2873599502,35cyclictest21536-21chrt12:11:263
2873599494,26cyclictest21162-21chrt10:22:453
2873599493,33cyclictest28088-21taskset08:36:073
2873599492,34cyclictest24189-21chrt09:33:423
2873599492,34cyclictest10328-21taskset09:11:083
28735994834,10cyclictest451rcuc/308:21:163
28735994826,15cyclictest451rcuc/307:56:183
2873599482,35cyclictest12985-21chrt11:03:223
2873599473,32cyclictest14977-21chrt10:12:033
2873599463,34cyclictest2910-21chrt08:56:243
2873599463,34cyclictest2910-21chrt08:56:243
2873599463,33cyclictest4258-21chrt07:21:273
2873599463,32cyclictest30756-21chrt11:32:233
28735994631,9cyclictest451rcuc/310:03:033
28735994631,9cyclictest451rcuc/310:03:033
2873599462,29cyclictest29999-21chrt10:36:473
2873599432,29cyclictest29153-21chrt07:01:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional