You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 17:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon Sep 14, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2848521748,57sleep228541-21latency07:04:452
2872021268,83sleep028113-21/usr/sbin/munin07:04:560
296099911718,26cyclictest18580-21sh12:14:050
29620991123,80cyclictest17378-21apt-get08:09:282
296099911145,61cyclictest29597-21cyclictest08:54:480
296099911016,20cyclictest14946-21sh12:07:110
29609991093,26cyclictest343-21runrttasks11:37:290
296139910651,49cyclictest29597-21cyclictest07:24:421
296099910345,51cyclictest29597-21cyclictest11:29:420
29609991034,12cyclictest21944-21apt-get12:19:240
29609991032,24cyclictest1482-21ssh11:44:370
296209910245,49cyclictest0-21swapper/212:29:252
296099910243,53cyclictest17463-21cpu08:09:500
296099910232,61cyclictest0-21swapper/009:09:490
296099910232,61cyclictest0-21swapper/009:09:490
296099910224,74cyclictest14650irq/292-2188000.ethernet08:59:260
29620991013,67cyclictest26516-21diskstats09:45:052
296099910144,49cyclictest0-21swapper/011:59:500
296099910124,55cyclictest0-21swapper/010:29:170
296209910022,42cyclictest1772-21sh10:50:092
296099910043,50cyclictest16086-21sh10:20:280
29609991003,30cyclictest1771-21taskset10:50:090
29609991002,22cyclictest4265-21ssh11:49:320
29609991002,22cyclictest4265-21ssh11:49:320
29609991002,20cyclictest14855-21kworker/u17:0+rpciod09:29:430
313122990,6sleep20-21swapper/211:39:482
313122990,6sleep20-21swapper/211:39:482
2960999993,40cyclictest4234-21date09:09:170
29609999930,62cyclictest0-21swapper/012:14:170
2960999983,27cyclictest343-21runrttasks08:45:350
2960999983,21cyclictest10637-21/usr/sbin/munin09:20:340
2960999983,11cyclictest0-21swapper/009:25:550
2960999973,32cyclictest24298-21modprobe11:29:130
2960999973,32cyclictest24298-21modprobe11:29:130
29609999639,49cyclictest0-21swapper/011:04:350
2960999963,38cyclictest17359-21seq11:16:310
2960999963,38cyclictest17359-21seq11:16:310
2960999963,38cyclictest17359-21seq11:16:310
29613999550,40cyclictest29597-21cyclictest10:54:411
29613999550,40cyclictest29597-21cyclictest10:54:411
2960999953,39cyclictest29342-21forks09:49:340
2960999953,24cyclictest21182-21/usr/sbin/munin10:29:330
2960999953,22cyclictest9309-21/usr/sbin/munin10:09:410
2960999952,29cyclictest7226-21grep10:59:470
217422950,7sleep30-21swapper/310:29:463
2962099942,49cyclictest3381-21taskset11:48:102
29609999440,46cyclictest0-21swapper/008:24:140
2960999943,82cyclictest7927-21netstat09:14:410
29609999436,51cyclictest0-21swapper/008:49:420
2960999943,23cyclictest1-21systemd07:59:150
2960999943,21cyclictest31218-21basename12:34:370
2960999942,42cyclictest28014-21ssh12:29:300
2960999942,32cyclictest30988-21/usr/sbin/munin11:39:390
2960999942,32cyclictest30988-21/usr/sbin/munin11:39:390
2960999942,21cyclictest3020-21rm09:59:340
2868729433,34sleep30-21swapper/307:04:553
2962699938,49cyclictest0-21swapper/309:45:013
29613999351,36cyclictest1154-21/usr/sbin/munin12:39:151
2960999934,30cyclictest6377-21latency_hist10:59:170
2960999934,30cyclictest6377-21latency_hist10:59:170
2960999932,24cyclictest25509-21/usr/sbin/munin08:34:310
2960999932,24cyclictest25509-21/usr/sbin/munin08:34:310
29609999315,25cyclictest6640-21ls10:04:500
29609999315,21cyclictest12580-21cat10:14:480
29613999239,44cyclictest0-21swapper/109:24:471
29609999236,49cyclictest0-21swapper/008:04:130
2960999923,41cyclictest20662-21ssh11:22:240
2960999923,30cyclictest9977-21grep07:44:500
2960999923,30cyclictest9977-21grep07:44:500
2960999923,29cyclictest27061-21/usr/sbin/munin08:39:460
2960999923,29cyclictest27061-21/usr/sbin/munin08:39:460
2960999923,20cyclictest16653-21chrt08:06:290
2960999922,24cyclictest20436-21rm09:34:420
2960999913,32cyclictest23482-21ntpq09:39:420
2960999913,31cyclictest3243-21grep07:24:470
2960999913,28cyclictest8793-21cron07:44:130
2960999913,28cyclictest8793-21cron07:44:130
2960999912,33cyclictest13597-21taskset11:09:580
2960999912,33cyclictest13597-21taskset11:09:580
2960999912,26cyclictest28453-21taskset10:40:490
29609999117,25cyclictest26898-21sh12:27:200
2962099902,75cyclictest18112-21ssh10:24:442
29613999045,38cyclictest21940-21sh12:19:241
2961399904,32cyclictest4088-21ssh11:49:261
2961399904,32cyclictest4088-21ssh11:49:261
29613999039,40cyclictest0-21swapper/110:04:361
29613999036,46cyclictest0-21swapper/108:04:171
2960999903,26cyclictest22219-21/usr/sbin/munin08:24:390
2960999903,25cyclictest762-21sendmail-msp07:19:140
2960999903,21cyclictest32357-21ssh10:48:210
2960999903,21cyclictest32357-21ssh10:48:210
2960999902,21cyclictest26592-21ssh10:39:050
29609999021,60cyclictest0-21swapper/009:59:170
29620998944,39cyclictest27701-21diskmemload12:29:162
29620998942,38cyclictest0-21swapper/211:59:502
2961399895,36cyclictest27477-21grep10:39:391
29613998932,12cyclictest0-21swapper/107:14:141
29613998932,12cyclictest0-21swapper/107:14:141
2960999893,20cyclictest29886-21cstates07:09:320
2960999893,20cyclictest29886-21cstates07:09:320
2960999892,29cyclictest26342-21ssh09:45:000
2960999892,25cyclictest7222-21/usr/sbin/munin11:54:320
2961399883,41cyclictest22875-21/usr/sbin/munin09:39:451
2961399883,34cyclictest29597-21cyclictest07:09:181
2961399883,34cyclictest22169-21ssh09:38:101
2961399883,32cyclictest16874-21grep12:09:471
29613998830,9cyclictest0-21swapper/111:34:491
29620998742,39cyclictest0-21swapper/207:14:132
29620998742,39cyclictest0-21swapper/207:14:132
2962099872,68cyclictest18990-21ssh12:14:292
2961399874,34cyclictest23926-21ls08:29:311
2961399874,31cyclictest7648-21fschecks_count07:39:361
2961399874,31cyclictest7648-21fschecks_count07:39:361
29613998740,42cyclictest0-21swapper/111:08:271
29613998733,8cyclictest0-21swapper/108:54:281
2961399873,36cyclictest24582-21/usr/sbin/munin11:29:461
2961399873,35cyclictest18727-21ssh11:19:321
2961399873,35cyclictest18727-21ssh11:19:321
2961399873,35cyclictest18727-21ssh11:19:321
2961399873,34cyclictest24318-21fschecks_time10:34:391
2961399873,32cyclictest16863-21/usr/sbin/munin09:29:581
2961399872,38cyclictest2537-21runrttasks07:24:171
2961399872,30cyclictest28213-21fschecks_time12:29:351
29613998721,12cyclictest0-21swapper/107:19:171
2960999873,28cyclictest7406-21grep07:39:180
2960999873,25cyclictest366-21sshd07:19:480
2960999873,25cyclictest19607-21sendmail_mailqu08:14:480
29609998710,22cyclictest12150-21latency_hist07:54:160
2962099864,70cyclictest17072-21ssh09:29:512
2962099863,59cyclictest15679-21ssh11:14:282
2962099863,59cyclictest15679-21ssh11:14:282
2962099863,59cyclictest15679-21ssh11:14:282
2961399865,33cyclictest7620-21fschecks_count09:14:351
2961399864,36cyclictest17448-21apt-get10:24:211
2961399863,33cyclictest8700-21/usr/sbin/munin10:09:501
2961399863,31cyclictest17633-21fschecks_time08:09:561
2961399863,31cyclictest16875-21latency_hist08:09:161
29613998631,9cyclictest0-21swapper/110:49:141
29613998631,9cyclictest0-21swapper/110:49:141
2961399862,34cyclictest22053-21latency_hist08:24:161
2960999863,22cyclictest4930-21wc07:29:470
2961399854,34cyclictest12271-21/usr/sbin/munin07:54:481
2961399854,31cyclictest5733-21latency_hist07:34:171
2961399853,38cyclictest343-21runrttasks09:23:411
2961399853,35cyclictest32463-21fschecks_time09:54:381
2961399853,34cyclictest29385-21rm09:49:351
2961399853,34cyclictest21366-21latency_hist11:24:151
2961399853,29cyclictest26352-21grep08:34:511
2961399853,29cyclictest26352-21grep08:34:511
29613998532,42cyclictest0-21swapper/109:05:281
29613998526,8cyclictest0-21swapper/111:44:431
2961399852,36cyclictest21609-21/usr/sbin/munin11:24:401
2961399852,36cyclictest21609-21/usr/sbin/munin11:24:401
2961399852,34cyclictest6655-21cstates10:59:321
2960999853,29cyclictest23997-21runrttasks08:29:340
2961399845,28cyclictest18940-21/usr/sbin/munin12:14:411
2961399844,39cyclictest23678-21latency_hist08:29:161
2961399843,35cyclictest343-21runrttasks08:47:581
2961399843,35cyclictest27401-21grep09:45:271
2961399843,33cyclictest343-21runrttasks07:52:321
2961399843,33cyclictest19163-21fschecks_count08:14:361
2961399843,32cyclictest15446-21grep11:14:171
2961399843,32cyclictest15446-21grep11:14:171
2961399843,31cyclictest6096-21grep07:34:381
2961399843,31cyclictest3211-21modprobe10:54:131
2961399842,36cyclictest20964-21cat10:29:231
2961399842,33cyclictest4613-21fschecks_count09:09:351
2961399842,33cyclictest4613-21fschecks_count09:09:351
2962699835,38cyclictest0-21swapper/310:24:413
2962099834,61cyclictest10814-21ssh09:19:522
29620998343,34cyclictest0-21swapper/212:04:352
29620998340,36cyclictest0-21swapper/210:19:172
29620998339,36cyclictest0-21swapper/209:04:362
2962099832,75cyclictest0-21swapper/210:29:282
2961399834,32cyclictest1075-21/usr/sbin/munin08:59:411
2961399834,29cyclictest2693-21/usr/sbin/munin09:59:441
2961399833,34cyclictest30561-21/usr/sbin/munin11:39:421
2961399833,34cyclictest30561-21/usr/sbin/munin11:39:421
2961399833,34cyclictest25220-21fschecks_time12:24:361
2961399833,33cyclictest9652-21grep07:44:461
2961399833,33cyclictest9652-21grep07:44:461
2962099823,46cyclictest29074-21sh09:49:192
2962099823,35cyclictest26779-21modprobe08:39:142
2962099823,35cyclictest26779-21modprobe08:39:142
2961399825,30cyclictest28403-21cron08:44:131
2961399825,30cyclictest28403-21cron08:44:131
2961399824,32cyclictest12992-21/usr/sbin/munin12:04:371
2961399823,36cyclictest17424-21ssh10:24:131
2961399823,35cyclictest14422-21apt10:19:141
2962099817,34cyclictest181rcu_preempt12:09:222
29620998138,32cyclictest0-21swapper/208:24:312
29620998135,38cyclictest0-21swapper/212:19:362
29620998127,41cyclictest0-21swapper/208:04:172
2961399813,35cyclictest10936-21timerandwakeup11:59:491
2961399813,30cyclictest7030-21/usr/sbin/munin11:54:371
2961399813,27cyclictest30362-21cpuspeed08:49:301
2962699803,64cyclictest16932-21chrt09:29:323
29620998025,45cyclictest0-21swapper/210:34:512
2962099802,41cyclictest7822-21kernelversion09:14:392
29620997936,35cyclictest0-21swapper/208:34:142
29620997935,36cyclictest0-21swapper/210:19:442
29620997933,37cyclictest0-21swapper/207:14:232
29620997932,37cyclictest0-21swapper/210:14:172
2962099783,46cyclictest4056-21diskmemload11:26:512
2962099783,46cyclictest4056-21diskmemload11:26:512
29620997832,38cyclictest0-21swapper/207:36:362
2962099782,68cyclictest0-21swapper/208:44:242
29620997730,39cyclictest0-21swapper/211:54:472
29620997636,32cyclictest0-21swapper/211:54:152
29620997636,32cyclictest0-21swapper/211:54:152
2962099763,59cyclictest15543-21sshd08:04:272
29620997635,34cyclictest0-21swapper/207:24:162
29620997633,33cyclictest0-21swapper/207:49:472
29620997614,37cyclictest381ktimers/210:03:502
2962099753,62cyclictest32003-21cstates08:54:302
29620997535,32cyclictest0-21swapper/207:59:172
2962099752,65cyclictest0-21swapper/209:09:252
2962099752,65cyclictest0-21swapper/209:09:252
2962099752,64cyclictest20948-21grep09:34:522
29620997455,13cyclictest343-21runrttasks11:34:272
29620997454,14cyclictest662-21cron08:19:142
2962099743,64cyclictest27079-21apt-get08:39:242
2962099743,64cyclictest27079-21apt-get08:39:242
2962099743,59cyclictest1100-21sshd08:59:282
29620997432,33cyclictest0-21swapper/207:29:362
29620997430,37cyclictest0-21swapper/209:55:062
29620997429,38cyclictest0-21swapper/207:24:432
2962099742,64cyclictest0-21swapper/211:09:252
2962099742,64cyclictest0-21swapper/211:09:252
2962699739,33cyclictest0-21swapper/311:14:413
2962699739,33cyclictest0-21swapper/311:14:413
2962699739,33cyclictest0-21swapper/311:14:413
29620997354,13cyclictest1413-21latency_hist12:39:182
2962099733,62cyclictest6508-21apt-get10:59:302
29620997232,33cyclictest0-21swapper/211:19:502
2962099722,64cyclictest0-21swapper/207:44:292
2962099722,64cyclictest0-21swapper/207:44:292
29620997223,44cyclictest381ktimers/211:29:292
311092710,5sleep30-21swapper/312:34:353
2962099713,60cyclictest29344-21sh10:43:192
2962099713,56cyclictest5958-21/usr/sbin/munin10:04:362
2962099713,53cyclictest20630-21apt-get08:19:292
29620997133,31cyclictest0-21swapper/208:49:482
2962099703,52cyclictest10029-21netstat11:04:412
29620997031,32cyclictest0-21swapper/209:24:282
2962099702,43cyclictest0-21swapper/210:57:432
2962099702,43cyclictest0-21swapper/210:57:432
29620997022,39cyclictest30515-21memory10:44:402
29620997022,39cyclictest30515-21memory10:44:402
2962099692,57cyclictest0-21swapper/207:39:302
2962099692,57cyclictest0-21swapper/207:39:302
2962099683,55cyclictest22897-21apt-get09:39:292
2962699639,27cyclictest0-21swapper/309:19:363
2962699614,30cyclictest0-21swapper/308:09:323
2962699593,39cyclictest1891-21chrt08:59:503
2962699583,41cyclictest4753-21chrt11:49:453
2962699583,41cyclictest4753-21chrt11:49:453
29626995624,22cyclictest3715-21chrt10:54:343
29626995624,22cyclictest3715-21chrt10:54:343
29626995620,24cyclictest30033-21chrt10:44:293
29626995620,24cyclictest30033-21chrt10:44:293
2962699553,40cyclictest10147-21taskset11:59:303
2962699553,35cyclictest11980-21chrt10:14:343
29626995521,21cyclictest23566-21chrt12:21:183
29626995518,25cyclictest1563-21taskset09:57:003
29626995514,27cyclictest8919-21taskset09:16:023
2962699543,37cyclictest28250-21taskset10:40:163
2962699543,35cyclictest28282-21taskset12:29:373
2962699543,34cyclictest1372-21chrt11:44:353
2962699542,37cyclictest9956-21chrt07:44:493
2962699542,37cyclictest9956-21chrt07:44:493
2962699542,35cyclictest15010-21taskset11:13:593
2962699542,35cyclictest15010-21taskset11:13:593
29626995422,20cyclictest5828-21chrt09:10:473
29626995422,20cyclictest5828-21chrt09:10:473
29626995416,25cyclictest21180-21taskset11:23:523
2962699534,36cyclictest27007-21chrt12:27:383
2962699533,35cyclictest7222-21chrt07:39:153
2962699533,33cyclictest4369-21chrt07:29:313
2962699533,32cyclictest897-21chrt10:49:363
2962699532,34cyclictest2880-21taskset09:04:363
29626995321,20cyclictest15689-21chrt08:04:343
29626995320,21cyclictest28690-21chrt08:44:273
29626995320,21cyclictest17515-21chrt10:24:153
29626995320,20cyclictest10942-21chrt10:12:563
29626995319,22cyclictest727-21taskset11:44:063
29626995319,22cyclictest727-21taskset11:44:063
29626995318,24cyclictest16159-21chrt12:09:313
29626995317,24cyclictest24627-21chrt11:29:273
2962699523,37cyclictest10249-21taskset11:04:473
2962699523,36cyclictest14040-21taskset07:59:313
2962699523,34cyclictest14993-21chrt09:25:313
2962699523,33cyclictest19823-21taskset09:34:183
2962699522,38cyclictest31442-21taskset07:14:203
2962699522,35cyclictest23645-21chrt08:29:163
29626995219,20cyclictest28192-21chrt08:41:273
29626995219,20cyclictest28192-21chrt08:41:273
29626995219,20cyclictest22183-21chrt11:24:413
29626995219,20cyclictest22183-21chrt11:24:413
29626995219,20cyclictest14271-21chrt12:05:173
29626995218,21cyclictest20823-21chrt12:17:003
2962699513,36cyclictest28813-21chrt11:35:153
2962699513,35cyclictest6264-21chrt10:04:423
2962699513,33cyclictest30903-21chrt08:49:453
2962699512,36cyclictest1528-21taskset07:19:433
29626995121,20cyclictest29796-21chrt07:09:303
29626995121,20cyclictest29796-21chrt07:09:303
29626995118,20cyclictest8762-21chrt11:56:343
29626995116,23cyclictest32447-21taskset08:54:433
2962699503,35cyclictest8639-21taskset11:02:453
2962699502,33cyclictest4668-21chrt10:02:113
29626995019,20cyclictest11062-21chrt07:49:393
29626995018,20cyclictest26620-21chrt08:37:143
29626995018,20cyclictest26620-21chrt08:37:143
29626995018,20cyclictest25268-21chrt10:35:213
29626995017,21cyclictest24974-21chrt09:42:333
29626995016,21cyclictest20639-21chrt08:19:283
2962699494,31cyclictest3990-21taskset07:29:143
29626994918,19cyclictest8616-21chrt07:41:543
29626994918,19cyclictest8616-21chrt07:41:543
29626994916,21cyclictest24290-21chrt08:29:413
29626994915,21cyclictest18991-21chrt08:14:263
29626994816,21cyclictest30777-21chrt09:51:213
29626994715,21cyclictest13283-21chrt07:54:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional