You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 22:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Wed Jun 24, 2026 12:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1585991254,35cyclictest31920-21apt-get11:23:360
1585991212,29cyclictest19676-21snmpd12:14:370
1585991163,28cyclictest14466-21sh09:05:520
1585991163,28cyclictest14466-21sh09:05:520
1585991133,26cyclictest26203-21ssh11:14:150
1597991103,73cyclictest19189-21apt-get07:48:581
1585991103,24cyclictest4610-21ssh11:31:580
1585991103,24cyclictest4610-21ssh11:31:580
1585991102,26cyclictest28888-21sh11:18:290
1585991083,25cyclictest3116-21ssh09:40:290
1585991083,25cyclictest3116-21ssh09:40:290
1585991083,24cyclictest12549-21ssh10:50:470
1585991072,25cyclictest22240-21sh09:19:010
1585991054,33cyclictest1586-21find06:58:300
1585991043,9cyclictest0-21swapper/010:33:380
1585991033,26cyclictest13997-21taskset10:53:450
1585991032,29cyclictest352-21runrttasks12:19:230
1585991032,28cyclictest18715-21ssh10:07:250
1585991032,28cyclictest18715-21ssh10:07:250
1597991023,80cyclictest1345-21apt-get06:58:421
1597991023,80cyclictest1345-21apt-get06:58:421
1585991024,29cyclictest24455-21vmstat12:04:020
15859910147,47cyclictest15023-21ssh11:48:550
15859910144,49cyclictest0-21swapper/010:59:000
1585991012,27cyclictest352-21runrttasks10:11:130
158599994,9cyclictest0-21swapper/006:59:030
158599994,9cyclictest0-21swapper/006:59:030
1585999949,45cyclictest1572-21cyclictest12:01:220
158599994,11cyclictest0-21swapper/011:08:360
158599994,11cyclictest0-21swapper/011:08:360
158599993,27cyclictest1672-21latency_hist10:33:270
158599993,27cyclictest1672-21latency_hist10:33:270
158599993,24cyclictest5571-21latency_hist12:28:270
158599993,24cyclictest14257-21cat11:48:280
158599984,25cyclictest19757-21ssh11:57:340
158599983,26cyclictest21882-21cut09:18:290
158599974,30cyclictest30976-21/usr/sbin/munin09:33:540
158599973,7cyclictest0-21swapper/008:59:010
158599964,9cyclictest0-21swapper/010:23:370
158599963,27cyclictest352-21runrttasks07:16:260
158599963,26cyclictest352-21runrttasks10:22:480
158599962,28cyclictest4456-21/usr/sbin/munin09:43:350
158599954,35cyclictest6054-21ntp_states11:33:550
1585999541,45cyclictest0-21swapper/008:33:370
158599953,9cyclictest0-21swapper/009:23:420
158599953,26cyclictest352-21runrttasks09:51:280
1585999521,57cyclictest0-21swapper/009:53:370
1585999521,57cyclictest0-21swapper/009:53:370
158599944,28cyclictest9077-21modprobe08:58:240
158599944,28cyclictest9077-21modprobe08:58:240
1585999436,51cyclictest161ktimers/007:18:400
158599943,28cyclictest10382-21/usr/sbin/munin07:23:550
158599943,26cyclictest19041-21kworker/u17:3+rpciod12:08:570
158599943,25cyclictest352-21runrttasks08:45:580
158599943,25cyclictest352-21runrttasks08:45:580
158599943,25cyclictest13923-21apt-get07:34:180
158599943,25cyclictest13923-21apt-get07:34:180
158599942,25cyclictest5306-21idleruntime-cro07:08:250
158599934,32cyclictest22419-21apt11:08:240
158599934,32cyclictest22419-21apt11:08:240
158599933,31cyclictest18979-21apt-config07:48:290
158599933,29cyclictest8427-21ssh11:38:340
158599933,28cyclictest352-21runrttasks08:41:360
158599933,28cyclictest22385-21apt-get10:13:350
158599933,28cyclictest22385-21apt-get10:13:350
158599933,27cyclictest8105-21fschecks_time10:43:450
158599932,32cyclictest24274-21tune2fs08:03:470
158599932,28cyclictest352-21runrttasks07:42:060
158599924,8cyclictest0-21swapper/007:28:370
158599924,32cyclictest17346-21grep09:10:090
158599923,32cyclictest7678-21latency_hist08:53:280
158599923,19cyclictest30666-21taskset09:33:240
158599922,34cyclictest6929-21idleruntime-cro07:13:250
1585999211,32cyclictest31841-21run-parts08:28:250
159799913,35cyclictest27496-21fschecks_time08:13:481
158599913,30cyclictest23757-21latency_hist08:03:240
158599913,29cyclictest7496-21latency_hist10:43:280
158599913,29cyclictest7496-21latency_hist10:43:280
158599913,29cyclictest19430-21seq07:49:410
158599913,27cyclictest27208-21grep08:13:280
158599913,27cyclictest27208-21grep08:13:280
158599903,28cyclictest20789-21/usr/sbin/munin07:53:510
158599903,28cyclictest20789-21/usr/sbin/munin07:53:510
158599902,27cyclictest32345-21fschecks_time08:28:460
1585999011,59cyclictest0-21swapper/010:03:260
158599894,29cyclictest29766-21unixbench_multi08:19:000
158599893,28cyclictest28734-21latency_hist08:18:270
1597998663,14cyclictest16559-21latency_hist10:58:271
1620998561,18cyclictest18850-21/usr/sbin/munin09:13:552
1597998560,16cyclictest352-21runrttasks11:04:031
1597998560,16cyclictest352-21runrttasks11:04:031
1597998448,8cyclictest181rcu_preempt10:29:091
1597998448,8cyclictest181rcu_preempt10:29:091
34028115,35sleep20-21swapper/206:53:552
1620998157,16cyclictest14100-21sh09:05:082
1620998157,16cyclictest14100-21sh09:05:082
162099804,53cyclictest26012-21munin-run12:08:232
162099796,55cyclictest2374-21systemd-journal11:13:432
1597997856,15cyclictest2374-21systemd-journal11:45:511
1597997854,14cyclictest17202-21apt-key07:43:281
1597997852,18cyclictest29254-21fschecks_time11:18:461
159799783,50cyclictest15941-21apt-get09:08:421
136627812,42sleep11383-21apt-config06:58:271
162099779,58cyclictest18664-21apt-get07:48:392
1597997757,15cyclictest1572-21cyclictest10:38:431
1597997757,15cyclictest1572-21cyclictest10:38:431
1597997757,14cyclictest14115-21fschecks_count07:34:471
1597997757,14cyclictest14115-21fschecks_count07:34:471
1620997650,12cyclictest0-21swapper/209:58:482
1597997658,12cyclictest2374-21systemd-journal08:13:251
1597997658,12cyclictest2374-21systemd-journal08:13:251
1597997654,14cyclictest2374-21systemd-journal11:33:121
1597997654,14cyclictest2374-21systemd-journal11:33:121
159799763,28cyclictest25969-21apt-get11:13:411
3243827512,38sleep30-21swapper/306:53:363
1620997558,10cyclictest473-21unixbench_multi11:24:022
1597997551,18cyclictest13407-21apt-key07:33:271
1597997551,16cyclictest7317-21kworker/u17:0-writeback10:43:591
1597997528,30cyclictest9091-21diskmemload11:23:551
1620997431,34cyclictest22831-21latency10:13:502
1620997431,34cyclictest22831-21latency10:13:502
1597997445,7cyclictest181rcu_preempt10:51:111
1597997445,20cyclictest2262-21interrupts10:33:491
1597997441,7cyclictest181rcu_preempt11:53:421
1597997422,25cyclictest31-21ksoftirqd/110:05:051
1597997422,25cyclictest31-21ksoftirqd/110:05:051
1597997419,24cyclictest31-21ksoftirqd/107:08:351
1620997357,10cyclictest300-21rs:main2
1620997353,14cyclictest298-21in:imuxsock10:52:512
1620997351,15cyclictest25611-21ssh10:18:482
1620997349,15cyclictest7265-21kworker/u17:3-rpciod09:00:202
1597997342,20cyclictest8492-21switchtime08:54:001
1597997342,20cyclictest8492-21switchtime08:54:001
1597997332,27cyclictest28762-21sed10:23:501
1597997252,14cyclictest29880-21ntp_kernel_err12:13:521
1597997234,8cyclictest291rcuc/107:13:481
159799723,21cyclictest17261-21latency_hist11:53:271
159799723,17cyclictest18679-21systemctl07:48:251
1597997228,10cyclictest0-21swapper/111:42:271
159799722,54cyclictest9086-21apt-get08:58:401
1620997152,12cyclictest2088-21fschecks_count10:33:452
162099713,48cyclictest4753-21fschecks_time09:43:472
162099713,37cyclictest8454-21open_inodes10:43:542
162099712,49cyclictest5819-21fschecks_time07:08:462
1597997136,11cyclictest14477-21ssh09:59:011
162099703,47cyclictest2553-21grep06:59:222
162099703,47cyclictest2553-21grep06:59:222
159799708,25cyclictest31-21ksoftirqd/110:14:371
159799708,25cyclictest31-21ksoftirqd/110:14:371
1597997056,9cyclictest32280-21/usr/sbin/munin12:18:461
1597997056,10cyclictest1572-21cyclictest09:51:511
1597997049,13cyclictest32476-21kernelversion08:28:491
1597997018,21cyclictest291rcuc/107:54:031
1597997018,21cyclictest291rcuc/107:54:031
1597997010,26cyclictest31-21ksoftirqd/109:36:351
163799693,51cyclictest31870-21taskset08:28:253
163799693,51cyclictest20562-21taskset07:53:253
1620996953,9cyclictest9177-21latency07:18:492
1620996946,11cyclictest27910-21kworker/u18:0-ext4-rsv-conversion09:53:492
1620996946,11cyclictest27910-21kworker/u18:0-ext4-rsv-conversion09:53:492
1597996955,8cyclictest1587-21cpuspeed09:38:411
1597996955,8cyclictest1587-21cpuspeed09:38:411
1597996935,9cyclictest181rcu_preempt12:06:311
1597996935,9cyclictest181rcu_preempt09:54:431
1597996935,9cyclictest181rcu_preempt09:54:431
159799693,45cyclictest29232-21latency_hist12:13:281
1597996934,10cyclictest1572-21cyclictest12:26:161
163799682,51cyclictest12257-21taskset09:03:223
1620996853,9cyclictest23332-21kworker/u17:1-rpciod11:28:412
1620996853,9cyclictest23332-21kworker/u17:1-rpciod11:28:412
1620996853,9cyclictest17889-21ntp_offset07:44:292
1620996853,11cyclictest1572-21cyclictest09:49:032
1620996852,10cyclictest277-21dbus-daemon11:48:242
1620996852,10cyclictest25043-21/usr/sbin/munin09:23:542
1620996851,10cyclictest28314-21/usr/sbin/munin10:23:352
1597996849,9cyclictest0-21swapper/108:18:401
1597996849,15cyclictest6781-21kworker/u17:2-ext4-rsv-conversion10:13:271
1597996842,14cyclictest181rcu_preempt09:04:561
1597996842,14cyclictest181rcu_preempt09:04:561
1597996822,21cyclictest31-21ksoftirqd/109:43:571
1597996811,28cyclictest31-21ksoftirqd/109:30:091
163799673,50cyclictest16814-21taskset07:43:243
163799673,49cyclictest19081-21chrt11:02:263
163799672,53cyclictest22266-21chrt09:19:053
1620996720,28cyclictest1542-21cpu09:38:412
1620996720,28cyclictest1542-21cpu09:38:412
159799676,25cyclictest31-21ksoftirqd/110:21:181
1597996747,10cyclictest0-21swapper/108:33:571
1597996746,15cyclictest352-21runrttasks08:38:511
163799663,50cyclictest27796-21chrt08:13:553
163799663,50cyclictest24183-21taskset11:09:563
163799663,50cyclictest24183-21taskset11:09:563
163799663,49cyclictest12805-21taskset09:03:343
163799663,49cyclictest12805-21taskset09:03:343
163799663,49cyclictest10706-21chrt09:53:443
163799663,49cyclictest10706-21chrt09:53:443
163799662,50cyclictest21330-21chrt11:05:223
163799662,50cyclictest21330-21chrt11:05:223
1620996649,10cyclictest0-21swapper/209:23:262
1620996648,11cyclictest0-21swapper/210:28:462
1620996648,11cyclictest0-21swapper/210:28:462
1620996616,38cyclictest15369-21wc07:38:282
1620996616,38cyclictest15369-21wc07:38:282
159799666,25cyclictest31-21ksoftirqd/109:24:531
1597996648,12cyclictest8349-21latency_hist11:38:281
1597996645,10cyclictest0-21swapper/107:18:591
1597996628,27cyclictest23388-21cat09:19:581
163799653,49cyclictest27851-21taskset10:22:573
163799652,49cyclictest12786-21taskset10:51:283
163799652,48cyclictest3922-21chrt07:03:393
1620996545,14cyclictest352-21runrttasks08:43:572
1620996545,14cyclictest352-21runrttasks08:43:572
1620996545,14cyclictest21686-21unixbench_multi07:54:032
1620996545,14cyclictest21686-21unixbench_multi07:54:032
1620996539,14cyclictest6897-21timerandwakeup08:48:582
162099653,43cyclictest8007-21fschecks_time08:53:472
162099653,43cyclictest8007-21fschecks_time08:53:472
1620996518,34cyclictest22404-21sort07:58:352
1597996520,35cyclictest20785-21diskmemload09:16:181
1597996511,25cyclictest31-21ksoftirqd/111:02:111
163799643,47cyclictest5319-21chrt12:27:593
163799643,47cyclictest4247-21taskset11:30:573
163799643,47cyclictest4247-21taskset11:30:573
163799643,47cyclictest22973-21chrt10:13:533
163799643,47cyclictest22973-21chrt10:13:533
163799642,50cyclictest2350-21taskset12:23:003
163799642,47cyclictest19406-21taskset10:08:343
1620996448,10cyclictest23505-21mailstats11:08:582
1620996448,10cyclictest23505-21mailstats11:08:582
1620996444,14cyclictest17595-21sh11:53:452
1620996444,10cyclictest0-21swapper/209:13:252
1620996443,12cyclictest0-21swapper/211:04:032
1620996443,12cyclictest0-21swapper/211:04:032
1620996440,15cyclictest3004-21df_inode12:23:422
162099643,46cyclictest29767-21unixbench_multi08:19:002
162099643,43cyclictest352-21runrttasks08:17:482
162099643,43cyclictest11293-21grep07:24:012
162099642,47cyclictest9091-21diskmemload11:18:462
162099642,40cyclictest31927-21timerandwakeup09:34:012
1597996441,8cyclictest181rcu_preempt08:43:341
1597996441,8cyclictest181rcu_preempt08:43:341
1597996435,22cyclictest7597-21cat08:53:271
1597996432,8cyclictest181rcu_preempt11:10:131
1597996432,8cyclictest181rcu_preempt11:10:131
163799633,48cyclictest16065-21chrt10:57:193
163799633,47cyclictest26743-21chrt09:25:423
163799633,47cyclictest16003-21chrt09:08:473
163799633,46cyclictest4858-21chrt10:38:383
163799633,46cyclictest4858-21chrt10:38:383
163799632,50cyclictest14052-21taskset11:48:233
163799632,49cyclictest13648-21chrt09:58:453
163799632,47cyclictest19717-21chrt09:13:583
163799632,46cyclictest7339-21chrt07:13:433
163799632,46cyclictest6611-21taskset07:09:343
1620996348,9cyclictest778-21timerwakeupswit12:19:002
1620996348,8cyclictest0-21swapper/207:13:402
1620996344,13cyclictest17125-21munin-run11:53:262
1620996344,12cyclictest0-21swapper/208:08:272
1620996344,11cyclictest0-21swapper/210:54:002
1620996342,13cyclictest0-21swapper/210:39:022
1620996342,13cyclictest0-21swapper/210:39:022
162099633,39cyclictest352-21runrttasks08:42:362
162099633,33cyclictest8321-21ssh11:38:272
162099633,33cyclictest13712-21cat07:33:292
162099632,36cyclictest19074-21sh10:08:252
162099632,36cyclictest19074-21sh10:08:252
159799636,25cyclictest31-21ksoftirqd/111:58:331
159799634,28cyclictest31-21ksoftirqd/107:58:471
163799623,46cyclictest9472-21chrt10:45:363
163799623,46cyclictest5915-21taskset09:45:003
163799623,46cyclictest27775-21taskset11:16:013
163799623,46cyclictest23515-21chrt08:00:303
163799623,46cyclictest21532-21taskset11:59:103
163799623,45cyclictest21798-21taskset07:54:063
163799623,45cyclictest21798-21taskset07:54:063
163799622,48cyclictest25822-21chrt08:08:443
163799622,48cyclictest25822-21chrt08:08:443
163799622,48cyclictest24043-21taskset12:03:563
163799622,48cyclictest10108-21chrt07:23:153
163799622,47cyclictest30527-21chrt09:33:003
163799622,47cyclictest2832-21taskset09:39:413
163799622,47cyclictest2832-21taskset09:39:413
163799622,46cyclictest19494-21chrt11:56:503
1637996211,40cyclictest13915-21chrt07:34:133
1637996211,40cyclictest13915-21chrt07:34:133
1620996244,12cyclictest11085-21ntpd11:38:592
162099623,39cyclictest29521-21diskstats12:13:432
162099622,38cyclictest4358-21sort07:03:542
1620996221,33cyclictest26355-21apt-get12:08:352
1597996213,19cyclictest301ktimers/107:23:401
163799613,45cyclictest5775-21chrt08:47:493
163799613,45cyclictest5775-21chrt08:47:493
163799613,45cyclictest24550-21chrt08:03:543
163799613,43cyclictest16792-21chrt10:03:483
163799613,43cyclictest16792-21chrt10:03:483
163799612,47cyclictest8803-21chrt09:49:423
163799612,47cyclictest475-21chrt09:36:253
163799612,47cyclictest2948-21chrt06:59:463
163799612,47cyclictest2948-21chrt06:59:463
163799612,46cyclictest6605-21chrt08:48:503
163799612,46cyclictest29614-21chrt11:18:573
163799612,46cyclictest17573-21chrt07:44:223
163799612,46cyclictest1684-21taskset11:27:123
1620996136,15cyclictest0-21swapper/208:28:562
162099613,42cyclictest20358-21/usr/sbin/munin11:58:522
1597996124,11cyclictest0-21swapper/107:07:271
163799602,46cyclictest906-21chrt08:32:223
163799602,46cyclictest14936-21chrt11:48:523
163799602,46cyclictest13276-21chrt07:32:183
163799602,45cyclictest359-21chrt10:30:073
163799602,45cyclictest359-21chrt10:30:073
163799602,45cyclictest3030-21chrt08:38:423
163799602,45cyclictest30140-21chrt08:22:383
163799602,45cyclictest28360-21taskset10:23:383
163799602,44cyclictest6664-21chrt11:34:213
1620996045,9cyclictest3494-21munin-node08:23:482
1620996038,12cyclictest0-21swapper/208:33:412
162099602,37cyclictest25893-21fschecks_time08:08:462
162099602,37cyclictest25893-21fschecks_time08:08:462
163799593,44cyclictest8818-21taskset11:38:473
163799593,41cyclictest0-21swapper/312:08:353
163799592,45cyclictest29529-21chrt12:13:443
163799592,44cyclictest2281-21chrt08:34:123
1620995937,15cyclictest0-21swapper/209:29:012
1597995938,11cyclictest0-21swapper/108:03:471
1597995916,31cyclictest30943-21needreboot08:23:531
163799582,44cyclictest3755-21chrt10:36:153
163799582,44cyclictest10377-21taskset07:23:313
1620995819,32cyclictest16899-21sh10:58:412
163799572,45cyclictest9011-21chrt08:57:413
163799572,45cyclictest9011-21chrt08:57:413
1620995737,13cyclictest0-21swapper/207:38:462
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional