You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 15:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Sat Jul 04, 2026 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
140089911537,71cyclictest0-21swapper/009:22:570
140089911121,10cyclictest0-21swapper/011:33:000
140119910943,35cyclictest20879-21diskmemload11:59:401
140119910943,35cyclictest20879-21diskmemload11:59:401
14008991064,28cyclictest15544-21runrttasks09:42:450
140089910636,52cyclictest0-21swapper/012:22:470
14008991053,24cyclictest14010-21latency_hist06:57:470
14008991053,24cyclictest14010-21latency_hist06:57:470
140119910488,9cyclictest0-21swapper/110:18:211
140119910488,9cyclictest0-21swapper/110:18:211
14008991044,26cyclictest14555-21ps12:23:150
14008991044,26cyclictest14555-21ps12:23:150
140089910435,50cyclictest0-21swapper/011:17:490
14008991033,30cyclictest27904-21expr10:58:010
14008991025,35cyclictest27453-21latency_hist10:02:460
140089910242,52cyclictest1871-21latency_hist12:02:460
140089910242,52cyclictest1871-21latency_hist12:02:460
140089910242,52cyclictest0-21swapper/012:03:050
140089910242,52cyclictest0-21swapper/012:03:050
14008991024,23cyclictest32294-21find11:04:420
140089910240,56cyclictest15572-21latency_hist10:37:450
14008991014,29cyclictest10778-21apt-get11:23:460
14008991014,29cyclictest10778-21apt-get11:23:460
140089910042,52cyclictest1-21systemd09:19:480
14014999976,16cyclictest19362-21latency_hist07:12:472
14014999974,19cyclictest10352-21apt-key11:22:462
1400899993,41cyclictest28267-21grep09:08:210
1400899993,36cyclictest6338-21latency_hist08:12:450
1400899993,30cyclictest31639-21unixbench_singl10:08:200
14008999915,30cyclictest277-21dbus-daemon09:07:450
14008999915,30cyclictest277-21dbus-daemon09:07:450
1400899984,38cyclictest24502-21ssh09:57:470
1400899984,38cyclictest24502-21ssh09:57:470
1400899984,32cyclictest12535-21latency_hist09:37:450
1400899984,24cyclictest2674-21taskset10:14:430
1400899983,27cyclictest28977-21ssh11:53:070
14008999828,7cyclictest0-21swapper/006:58:190
14008999824,13cyclictest0-21swapper/011:38:200
1400899975,32cyclictest8461-21fschecks_time08:18:060
14008999726,8cyclictest15350irq/295-2188000.ethernet11:28:060
14008999646,44cyclictest17460-21sshd10:53:490
1400899964,35cyclictest27973-21chrt11:51:510
1400899964,33cyclictest12421-21grep08:28:200
1400899964,33cyclictest12421-21grep08:28:200
1400899964,27cyclictest25883-21latency_hist07:32:440
14008999638,53cyclictest13996-21cyclictest12:13:010
1400899963,32cyclictest352-21runrttasks11:16:080
1400899963,31cyclictest22598-21/usr/sbin/munin11:43:250
1400899963,31cyclictest21174-21/usr/sbin/munin08:58:230
1400899963,26cyclictest17182-21grep08:43:190
1400899953,28cyclictest18107-21fschecks_time07:08:050
14008999529,8cyclictest0-21swapper/007:23:170
14008999514,26cyclictest19451-21ntp_kernel_pll_10:43:130
14008999514,26cyclictest19451-21ntp_kernel_pll_10:43:130
14011999454,34cyclictest29402-21kworker/u17:2+events_unbound10:07:431
1400899945,28cyclictest1465-21/usr/sbin/munin11:08:040
1400899944,32cyclictest31208-21fschecks_time07:48:050
1400899943,33cyclictest30156-21runrttasks10:07:300
1400899943,27cyclictest7092-21cpuspeed10:23:010
1400899943,25cyclictest18166-21grep08:48:060
1400899943,25cyclictest18166-21grep08:48:060
14008999430,59cyclictest0-21swapper/009:13:150
14008999430,59cyclictest0-21swapper/009:13:150
1400899942,29cyclictest1-21systemd08:57:440
14008999421,9cyclictest0-21swapper/007:53:170
1400899935,41cyclictest6861-21/usr/sbin/munin09:28:000
1400899935,29cyclictest16121-21fschecks_count10:38:040
1400899935,29cyclictest16121-21fschecks_count10:38:040
1400899934,25cyclictest9969-21/usr/sbin/munin10:28:120
1400899934,25cyclictest9969-21/usr/sbin/munin10:28:120
1400899934,25cyclictest352-21runrttasks08:38:170
14008999337,49cyclictest0-21swapper/010:52:460
14008999337,49cyclictest0-21swapper/010:52:460
14008999335,51cyclictest0-21swapper/007:13:210
1400899933,32cyclictest30817-21latency_hist07:47:460
14008999326,8cyclictest0-21swapper/007:58:180
125092936,71sleep10-21swapper/106:52:511
125092936,71sleep10-21swapper/106:52:511
14014999267,14cyclictest0-21swapper/209:23:402
1401199923,76cyclictest0-21swapper/107:17:581
1400899924,29cyclictest16652-21grep09:43:200
1400899924,29cyclictest16652-21grep09:43:200
1400899924,26cyclictest4289-21/usr/sbin/munin10:18:050
1400899924,26cyclictest4289-21/usr/sbin/munin10:18:050
1400899924,26cyclictest10438-21/usr/sbin/munin08:23:160
1400899924,26cyclictest10438-21/usr/sbin/munin08:23:160
1400899923,34cyclictest13175-21cstates08:33:010
1400899923,30cyclictest21997-21grep07:18:200
1400899923,26cyclictest15884-21munin-node07:02:500
14008999226,8cyclictest0-21swapper/008:03:150
1400899922,34cyclictest5316-21fschecks_time12:08:050
14014999168,13cyclictest0-21swapper/211:52:472
14014999167,14cyclictest0-21swapper/207:58:042
1401199913,50cyclictest0-21swapper/107:07:451
1400899914,29cyclictest26773-21cat07:33:170
1400899914,29cyclictest26773-21cat07:33:170
1400899914,17cyclictest20879-21diskmemload09:48:190
1400899913,32cyclictest6745-21df08:13:010
1400899913,31cyclictest27739-21sort07:37:530
1400899913,31cyclictest27739-21sort07:37:530
14014999066,12cyclictest0-21swapper/207:07:472
14014999056,27cyclictest0-21swapper/208:52:472
14014999056,27cyclictest0-21swapper/208:52:472
14014999052,32cyclictest7111-21ntp_kernel_pll_08:13:112
14011999046,38cyclictest10463-21latency_hist11:22:461
1401199903,76cyclictest0-21swapper/108:17:571
14014998969,9cyclictest0-21swapper/208:37:462
14014998969,10cyclictest0-21swapper/210:23:082
14014998949,33cyclictest0-21swapper/212:07:472
14014998941,41cyclictest352-21runrttasks10:55:352
1401199893,76cyclictest0-21swapper/111:12:561
1401199892,82cyclictest0-21swapper/110:57:561
1401199892,7cyclictest0-21swapper/110:07:581
1401199892,77cyclictest0-21swapper/111:32:551
14014998873,9cyclictest1442-21runrttasks10:13:092
14014998868,9cyclictest0-21swapper/207:52:462
14014998866,12cyclictest0-21swapper/207:02:462
14011998846,36cyclictest21355-21cron09:52:431
14011998843,39cyclictest18388-21latency_hist09:47:461
14011998843,39cyclictest18388-21latency_hist09:47:461
1401199883,78cyclictest0-21swapper/109:02:541
1401199883,78cyclictest0-21swapper/109:02:541
1401199873,71cyclictest0-21swapper/108:27:581
1401199873,71cyclictest0-21swapper/108:27:581
1401199873,70cyclictest0-21swapper/107:32:561
1401199873,70cyclictest0-21swapper/107:32:561
1269728711,39sleep20-21swapper/206:53:092
1269728711,39sleep20-21swapper/206:53:092
14014998667,10cyclictest0-21swapper/207:27:462
14014998664,11cyclictest0-21swapper/207:37:462
14014998664,11cyclictest0-21swapper/207:37:462
14014998638,40cyclictest0-21swapper/208:47:432
1401499863,53cyclictest21074-21apt-get07:17:562
14014998631,48cyclictest0-21swapper/211:03:392
14011998665,13cyclictest297-21systemd-logind09:37:441
14011998644,35cyclictest25229-21sendmail_mailqu07:29:161
14011998643,36cyclictest25436-21sh10:53:161
1271728611,53sleep30-21swapper/306:53:103
1271728611,53sleep30-21swapper/306:53:103
14014998537,40cyclictest0-21swapper/209:29:062
14014998530,49cyclictest0-21swapper/209:13:062
14014998530,49cyclictest0-21swapper/209:13:062
14014998515,26cyclictest15859-21cpuspeed09:43:002
14014998515,26cyclictest15859-21cpuspeed09:43:002
1401199856,67cyclictest14672-21apt-get08:37:561
1401199853,69cyclictest0-21swapper/107:37:551
1401199853,69cyclictest0-21swapper/107:37:551
1401199853,68cyclictest0-21swapper/108:47:561
1401199853,68cyclictest0-21swapper/108:47:561
14014998469,8cyclictest0-21swapper/210:58:182
14014998464,9cyclictest0-21swapper/208:53:052
1401499844,27cyclictest19432-21/usr/sbin/munin10:43:132
1401499844,27cyclictest19432-21/usr/sbin/munin10:43:132
1401499843,41cyclictest9084-21ssh12:13:232
1401499843,40cyclictest2188-21/usr/sbin/munin11:08:182
1401499843,26cyclictest13623-21latency_hist12:22:452
1401499843,24cyclictest15354-21/usr/sbin/munin08:38:182
14014998417,28cyclictest2054-21diskmemload09:19:022
14011998438,38cyclictest0-21swapper/107:58:041
1401199843,68cyclictest0-21swapper/106:58:001
14014998363,10cyclictest0-21swapper/207:57:472
14014998342,35cyclictest20756-21sh11:38:212
14014998340,36cyclictest0-21swapper/212:23:112
14014998340,36cyclictest0-21swapper/212:23:112
1401499833,40cyclictest27535-21sh10:02:472
14014998313,22cyclictest24708-21seq11:46:032
14014998241,34cyclictest11544-21sh09:35:052
14014998240,35cyclictest0-21swapper/208:58:142
14014998235,39cyclictest0-21swapper/210:28:202
14014998235,39cyclictest0-21swapper/210:28:202
14014998235,39cyclictest0-21swapper/209:48:192
14014998234,42cyclictest30395-21diskmemload11:55:202
1401499823,26cyclictest1019-21latency_hist10:12:472
1401199823,73cyclictest0-21swapper/108:42:561
1401499813,39cyclictest4566-21sh10:18:112
1401499813,39cyclictest4566-21sh10:18:112
1401499813,29cyclictest12764-21/usr/sbin/munin09:38:092
1401499813,26cyclictest22006-21cat10:48:012
1401499813,26cyclictest22006-21cat10:48:012
14014998131,43cyclictest0-21swapper/211:37:452
14014998119,24cyclictest27835-21cstates07:37:592
14014998119,24cyclictest27835-21cstates07:37:592
14014998116,23cyclictest13371-21cut10:33:102
1401199813,73cyclictest0-21swapper/109:07:541
1401199812,72cyclictest735-21ssh09:17:591
1401199812,6cyclictest0-21swapper/110:37:581
1401199812,6cyclictest0-21swapper/110:37:581
1401199812,69cyclictest0-21swapper/107:47:561
1401199812,46cyclictest10554-21/usr/sbin/munin10:28:131
1401199812,46cyclictest10554-21/usr/sbin/munin10:28:131
1401499804,37cyclictest26416-21ssh09:06:302
1401499804,37cyclictest26416-21ssh09:06:302
14014998036,38cyclictest0-21swapper/208:08:192
1401499803,35cyclictest10756-21/usr/sbin/munin11:24:102
1401499803,35cyclictest10756-21/usr/sbin/munin11:24:102
1401499803,25cyclictest28157-21timerwakeupswit09:08:202
1401499803,25cyclictest22523-21/usr/sbin/munin09:53:192
1401499803,25cyclictest22523-21/usr/sbin/munin09:53:192
1401499803,24cyclictest31534-21/usr/sbin/munin11:58:212
1401499803,24cyclictest31534-21/usr/sbin/munin11:58:212
1401499803,24cyclictest18543-21latency_hist10:42:462
1401499803,24cyclictest18543-21latency_hist10:42:462
1401199803,72cyclictest0-21swapper/109:52:551
1401199803,72cyclictest0-21swapper/109:52:551
1401199802,7cyclictest0-21swapper/110:42:591
1401199802,7cyclictest0-21swapper/110:42:591
1401199802,73cyclictest0-21swapper/109:12:581
1401199802,73cyclictest0-21swapper/109:12:581
14014997960,14cyclictest5327-21timerandwakeup11:13:202
1401499793,28cyclictest24068-21http07:27:482
1401499793,25cyclictest24928-21ssh09:58:052
1401499793,24cyclictest11442-21/usr/sbin/munin08:27:542
1401499793,24cyclictest11442-21/usr/sbin/munin08:27:542
14014997931,43cyclictest0-21swapper/208:23:102
14014997931,43cyclictest0-21swapper/208:23:102
14014997928,44cyclictest0-21swapper/208:03:202
1401499792,24cyclictest29975-21grep07:43:172
14014997910,22cyclictest15626-21diskmemload11:30:322
1401199793,69cyclictest0-21swapper/111:27:591
1401199792,6cyclictest0-21swapper/111:23:481
1401199792,6cyclictest0-21swapper/111:23:481
1401499783,28cyclictest2668-21taskset12:03:132
1401499783,28cyclictest2668-21taskset12:03:132
14011997838,33cyclictest0-21swapper/112:18:191
14011997837,33cyclictest0-21swapper/108:58:011
14011997835,36cyclictest0-21swapper/108:23:191
14011997835,36cyclictest0-21swapper/108:23:191
14011997835,33cyclictest0-21swapper/107:53:071
14011997826,42cyclictest7064-21df_inode09:28:021
1401499773,30cyclictest8795-21processes08:18:172
14011997740,32cyclictest0-21swapper/110:13:241
14011997737,33cyclictest14709-21diskmemload09:40:411
14011997737,33cyclictest0-21swapper/112:15:001
1401199773,63cyclictest0-21swapper/107:43:001
14011997736,34cyclictest0-21swapper/110:34:441
1401199772,63cyclictest0-21swapper/108:32:551
1401499763,25cyclictest19913-21/usr/sbin/munin07:13:102
1401199763,68cyclictest0-21swapper/111:03:391
1401199763,67cyclictest0-21swapper/108:12:591
1401199763,61cyclictest0-21swapper/107:07:541
1401199763,56cyclictest23209-21ntpq11:43:131
14011997542,26cyclictest6717-21sh09:27:461
14011997537,31cyclictest0-21swapper/109:58:201
1401199753,61cyclictest23197-21awk07:23:121
1401199753,60cyclictest0-21swapper/107:12:551
14011997533,35cyclictest20879-21diskmemload10:49:161
14011997533,35cyclictest20879-21diskmemload10:49:161
14011997530,37cyclictest0-21swapper/110:22:541
14011997529,39cyclictest0-21swapper/108:03:011
14011997435,32cyclictest0-21swapper/112:03:101
14011997435,32cyclictest0-21swapper/112:03:101
1401199742,6cyclictest0-21swapper/108:07:571
14011997426,41cyclictest0-21swapper/112:23:191
14011997426,41cyclictest0-21swapper/112:23:191
1401199733,63cyclictest0-21swapper/112:07:541
1401199723,62cyclictest4353-21ssh11:12:471
14011997233,32cyclictest0-21swapper/111:38:071
14011997132,33cyclictest27117-21sh11:49:251
1401199712,53cyclictest28770-21ssh11:53:011
1401199693,56cyclictest19526-21apt-get08:53:001
1402099682,25cyclictest433-21chrt09:17:453
1402099682,25cyclictest433-21chrt09:17:453
1402099673,30cyclictest18534-21chrt09:47:463
1402099673,30cyclictest18534-21chrt09:47:463
1402099673,25cyclictest11723-21chrt12:18:183
1402099663,27cyclictest23530-21chrt09:01:483
1402099663,26cyclictest9703-21taskset10:27:443
1402099662,25cyclictest23297-21taskset09:54:483
1402099662,25cyclictest23297-21taskset09:54:483
1402099659,45cyclictest19064-21chrt10:43:053
1402099659,45cyclictest19064-21chrt10:43:053
1402099653,26cyclictest16535-21taskset12:27:433
1402099653,26cyclictest16535-21taskset12:27:433
1402099653,26cyclictest13935-21chrt11:28:023
1402099653,25cyclictest3584-21chrt08:03:063
1402099653,22cyclictest18831-21chrt09:47:583
1402099652,30cyclictest26484-21taskset10:00:273
1402099652,27cyclictest3465-21taskset12:04:153
1402099652,27cyclictest3465-21taskset12:04:153
1402099652,27cyclictest20665-21taskset11:38:203
1402099652,26cyclictest23903-21taskset07:26:263
1402099652,24cyclictest355-21taskset11:05:313
1402099643,22cyclictest10020-21chrt09:33:023
1402099642,28cyclictest23381-21taskset11:43:193
1402099642,27cyclictest6505-21taskset09:27:393
1402099642,22cyclictest24988-21chrt10:53:023
1402099633,24cyclictest7440-21chrt12:12:053
1402099632,23cyclictest5284-21chrt08:08:083
1402099632,23cyclictest30921-21chrt10:08:063
1402099632,22cyclictest24522-21chrt07:27:523
1402099624,21cyclictest17330-21chrt11:33:133
1402099623,27cyclictest9922-21chrt11:21:543
1402099623,25cyclictest2832-21taskset11:09:043
1402099622,28cyclictest12968-21chrt09:38:013
1402099622,26cyclictest5526-21chrt11:13:223
1402099622,25cyclictest32573-21taskset07:52:493
1402099622,24cyclictest28369-21chrt10:58:133
1402099622,24cyclictest26196-21chrt07:33:003
1402099622,24cyclictest26196-21chrt07:33:003
1402099622,24cyclictest12847-21chrt11:26:073
1402099622,24cyclictest12847-21chrt11:26:073
1402099622,23cyclictest1719-21chrt10:13:173
1402099622,22cyclictest8294-21taskset12:13:073
1402099622,22cyclictest27373-21chrt11:50:083
1402099622,22cyclictest18321-21taskset10:42:073
1402099622,22cyclictest18321-21taskset10:42:073
1402099622,21cyclictest22240-21chrt07:20:393
1402099622,20cyclictest14596-21chrt06:58:103
1402099613,26cyclictest13124-21chrt10:33:033
1402099613,25cyclictest31584-21chrt11:57:583
1402099613,25cyclictest31584-21chrt11:57:583
1402099613,22cyclictest13534-21chrt08:33:123
1402099612,26cyclictest20450-21taskset07:13:213
1402099612,25cyclictest10623-21chrt10:28:173
1402099612,25cyclictest10623-21chrt10:28:173
1402099612,24cyclictest30121-21chrt11:54:343
1402099612,23cyclictest30536-21chrt07:45:523
1402099612,22cyclictest8728-21chrt08:18:133
1402099612,22cyclictest7650-21chrt08:14:043
1402099602,27cyclictest28996-21taskset07:41:433
1402099602,27cyclictest28996-21taskset07:41:433
1402099602,26cyclictest15901-21taskset08:41:303
1402099602,25cyclictest18058-21chrt08:48:023
1402099602,25cyclictest18058-21chrt08:48:023
1402099602,23cyclictest26779-21chrt09:07:323
1402099602,23cyclictest26779-21chrt09:07:323
1402099602,23cyclictest23120-21chrt10:48:493
1402099602,23cyclictest23120-21chrt10:48:493
1402099602,22cyclictest6634-21taskset10:22:373
1402099602,22cyclictest6634-21taskset10:22:373
1402099602,21cyclictest28688-21chrt09:09:263
1402099602,21cyclictest2276-21chrt07:58:123
1402099602,21cyclictest17865-21chrt07:07:553
1402099602,21cyclictest17425-21chrt07:06:043
1402099593,22cyclictest7470-21chrt09:28:123
1402099592,25cyclictest30321-21chrt10:07:453
1402099592,24cyclictest724-21chrt09:17:583
1402099592,21cyclictest20839-21chrt08:57:343
1402099583,24cyclictest32070-21taskset07:49:293
1402099583,24cyclictest10814-21chrt08:23:273
1402099583,24cyclictest10814-21chrt08:23:273
1402099582,25cyclictest12692-21taskset08:31:383
1402099582,25cyclictest12692-21taskset08:31:383
1402099582,24cyclictest17268-21chrt08:43:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional