You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-11-14 - 02:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Fri Nov 13, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
176619911145,12cyclictest0-21swapper/209:19:402
176619910960,13cyclictest1425-21sh11:11:132
176619910960,13cyclictest1425-21sh11:11:132
17655991023,79cyclictest23260-21ssh10:54:311
17655991023,79cyclictest23260-21ssh10:54:311
1634729615,53sleep216176-21/usr/sbin/munin07:04:272
17648999513,62cyclictest8050-21kworker/u19:0+flush-179:010:54:330
17648999513,62cyclictest8050-21kworker/u19:0+flush-179:010:54:330
17655999251,33cyclictest0-21swapper/109:19:271
17655999222,42cyclictest24208-21seq07:25:501
17655999153,32cyclictest431-21fschecks_count12:04:271
17655999153,32cyclictest431-21fschecks_count12:04:271
1765599913,45cyclictest4309-21ssh10:22:521
1765599913,45cyclictest4309-21ssh10:22:521
1764899912,47cyclictest28780-21modprobe11:04:050
17648999112,58cyclictest15-21ksoftirqd/011:54:080
17661999066,16cyclictest27012-21fschecks_time11:54:272
17661999066,16cyclictest27012-21fschecks_time11:54:272
17661999064,17cyclictest278-21dbus-daemon12:19:072
17661999010,72cyclictest8050-21kworker/u19:0+flush-179:010:54:262
17661999010,72cyclictest8050-21kworker/u19:0+flush-179:010:54:262
1765599903,72cyclictest0-21swapper/107:49:181
17655999023,36cyclictest0-21swapper/111:35:511
17648999015,46cyclictest15-21ksoftirqd/008:54:370
17648999015,46cyclictest15-21ksoftirqd/008:54:370
1765599893,67cyclictest5018-21sh09:30:121
1765599893,67cyclictest5018-21sh09:30:121
17655998865,16cyclictest18611-21sh11:39:391
1765599883,71cyclictest0-21swapper/112:34:181
1765599883,71cyclictest0-21swapper/112:34:181
17655998823,34cyclictest0-21swapper/109:59:061
17648998859,17cyclictest31652-21ssh10:14:210
17648998849,9cyclictest181rcu_preempt10:50:090
1766999873,61cyclictest0-21swapper/310:54:223
1766999873,61cyclictest0-21swapper/310:54:223
17661998647,11cyclictest0-21swapper/210:01:372
1766199863,73cyclictest9316-21apt-get12:19:252
1766199863,56cyclictest15230-21ssh11:35:222
1765599863,70cyclictest0-21swapper/109:34:191
1765599863,50cyclictest1900-21seq07:59:031
59942850,11sleep05998-21latency_hist12:14:080
17661998555,21cyclictest13097-21sh10:37:292
17655998563,15cyclictest31770-21cut10:14:261
17655998546,32cyclictest0-21swapper/111:34:081
1765599854,54cyclictest16542-21modprobe09:49:091
1765599854,54cyclictest16542-21modprobe09:49:091
1765599853,50cyclictest1444-21apt-get09:24:141
17655998526,33cyclictest0-21swapper/110:13:441
17655998522,36cyclictest2727-21/usr/sbin/munin11:14:411
17648998565,14cyclictest23525-21ssh09:59:420
1764899854,47cyclictest15-21ksoftirqd/009:20:240
1764899852,47cyclictest15-21ksoftirqd/009:16:050
17655998442,36cyclictest24550-21diskmemload09:14:291
17655998438,39cyclictest0-21swapper/109:09:331
1765599843,67cyclictest21438-21latency_hist07:19:081
1765599843,67cyclictest21438-21latency_hist07:19:081
1764899845,41cyclictest15-21ksoftirqd/011:09:310
1764899845,41cyclictest15-21ksoftirqd/011:09:310
1764899844,47cyclictest15-21ksoftirqd/010:44:210
17661998342,21cyclictest28591-21ssh09:14:392
17655998331,29cyclictest0-21swapper/111:56:371
17655998331,29cyclictest0-21swapper/111:56:371
17655998327,46cyclictest7778-21grep10:29:071
17655998240,35cyclictest0-21swapper/112:19:171
1765599823,48cyclictest19674-21latency_hist10:49:081
17648998265,11cyclictest17638-21cyclictest09:39:400
17648998265,11cyclictest17638-21cyclictest09:39:400
1764899822,46cyclictest15-21ksoftirqd/007:44:020
1766199815,51cyclictest5081-21sort09:30:172
1766199815,51cyclictest5081-21sort09:30:172
17661998154,19cyclictest20825-21unixbench_singl07:14:442
17661998154,19cyclictest20825-21unixbench_singl07:14:442
17661998151,13cyclictest7016-21kworker/u18:1-rpciod08:14:422
17661998151,13cyclictest7016-21kworker/u18:1-rpciod08:14:422
17655998135,39cyclictest0-21swapper/112:19:081
17655998133,41cyclictest0-21swapper/110:39:101
17648998161,14cyclictest19418-21munin-node12:34:460
17648998161,14cyclictest19418-21munin-node12:34:460
17648998111,46cyclictest15-21ksoftirqd/010:39:310
1766999804,32cyclictest22611-21chrt09:59:183
17661998065,9cyclictest23244-21kworker/u17:1-rpciod11:04:242
17655998058,16cyclictest3438-21cut12:09:281
17655998038,36cyclictest0-21swapper/111:14:091
17655998038,36cyclictest0-21swapper/111:14:091
17655998037,37cyclictest14260-21open_files09:44:371
1765599803,65cyclictest0-21swapper/108:09:171
1765599802,66cyclictest0-21swapper/108:14:201
1765599802,66cyclictest0-21swapper/108:14:201
17648998038,30cyclictest9068-21cat08:19:300
1764899803,44cyclictest15-21ksoftirqd/011:27:410
17648998026,29cyclictest15-21ksoftirqd/008:14:080
1764899802,47cyclictest15-21ksoftirqd/011:46:510
1764899802,44cyclictest15-21ksoftirqd/012:15:590
17648998020,37cyclictest15-21ksoftirqd/007:25:130
17648998010,44cyclictest15-21ksoftirqd/010:12:470
17661997955,17cyclictest27881-21latency_hist07:39:092
17655997955,17cyclictest29650-21/usr/sbin/munin11:59:361
1765599793,62cyclictest0-21swapper/107:59:211
1765599792,62cyclictest4721-21kworker/u19:3+ext4-rsv-conversion11:04:281
17648997955,10cyclictest181rcu_preempt07:09:260
1764899794,52cyclictest15-21ksoftirqd/010:34:260
1764899793,48cyclictest15-21ksoftirqd/011:19:230
17648997930,34cyclictest17638-21cyclictest07:49:300
17648997912,31cyclictest15-21ksoftirqd/009:30:260
17648997912,31cyclictest15-21ksoftirqd/009:30:260
17661997857,15cyclictest26384-21idleruntime-cro11:54:052
17661997856,14cyclictest24550-21diskmemload10:19:082
17655997835,36cyclictest0-21swapper/108:29:401
1765599783,51cyclictest357-21runrttasks07:21:001
17648997853,10cyclictest181rcu_preempt08:07:160
17648997850,9cyclictest181rcu_preempt08:44:290
17661997760,11cyclictest14344-21kworker/u17:3-rpciod12:32:072
17661997757,14cyclictest2235-21sh12:07:242
17661997757,14cyclictest2235-21sh12:07:242
17661997753,17cyclictest12564-21sh11:30:122
17661997751,16cyclictest357-21runrttasks09:42:512
17661997751,16cyclictest357-21runrttasks09:42:512
17661997731,15cyclictest181rcu_preempt11:14:392
17655997743,28cyclictest10695-21latency_hist10:34:081
1765599773,45cyclictest22295-21ssh11:45:521
17648997753,14cyclictest8682-21ssh10:29:360
1764899774,47cyclictest15-21ksoftirqd/008:27:580
1764899774,37cyclictest17540-21latency_hist07:09:090
1764899773,62cyclictest29329-21apt-get11:04:220
1764899773,38cyclictest15-21ksoftirqd/009:24:380
1764899772,37cyclictest15-21ksoftirqd/009:47:110
17661997641,26cyclictest8382-21modprobe11:24:052
1765599764,48cyclictest357-21runrttasks08:38:141
17655997637,32cyclictest0-21swapper/112:33:261
1765599763,48cyclictest357-21runrttasks10:01:431
1764899769,45cyclictest15-21ksoftirqd/010:24:220
1764899765,26cyclictest15-21ksoftirqd/009:04:420
1764899763,48cyclictest15-21ksoftirqd/011:55:410
1764899763,48cyclictest15-21ksoftirqd/011:55:410
1764899763,42cyclictest15-21ksoftirqd/007:22:220
1764899763,39cyclictest15-21ksoftirqd/012:24:070
17648997616,41cyclictest13313-21cat08:34:080
1766199753,45cyclictest12529-21ssh12:24:272
17655997549,18cyclictest357-21runrttasks12:29:081
1765599753,65cyclictest9474-21timerandwakeup08:19:401
1765599753,60cyclictest0-21swapper/107:39:171
1765599753,58cyclictest19957-21apt-get08:54:201
1765599753,58cyclictest19957-21apt-get08:54:201
1765599753,57cyclictest11550-21ssh11:29:081
17655997535,32cyclictest0-21swapper/110:49:341
17655997531,37cyclictest0-21swapper/109:00:301
1765599752,61cyclictest0-21swapper/108:49:171
17648997548,19cyclictest1-21systemd08:49:240
1764899754,33cyclictest15-21ksoftirqd/008:59:320
1764899753,63cyclictest29847-21modprobe07:44:330
1764899753,59cyclictest19276-21modprobe09:54:050
1764899753,59cyclictest19276-21modprobe09:54:050
1764899753,34cyclictest15-21ksoftirqd/011:59:200
1764899752,60cyclictest20143-21fschecks_time07:14:290
1764899752,60cyclictest20143-21fschecks_time07:14:290
17648997521,32cyclictest15-21ksoftirqd/008:34:400
72012742,20sleep07270-21fschecks_time08:14:270
72012742,20sleep07270-21fschecks_time08:14:270
17661997450,11cyclictest0-21swapper/210:04:272
1766199744,51cyclictest9304-21wc08:19:372
1766199744,26cyclictest0-21swapper/207:14:082
1766199742,59cyclictest29818-21apt-get07:44:332
17655997454,13cyclictest3390-21kworker/u17:1-rpciod09:43:181
17655997454,13cyclictest3390-21kworker/u17:1-rpciod09:43:181
17655997450,15cyclictest25087-21sh11:50:251
17655997448,20cyclictest15599-21netstat08:39:311
1765599743,62cyclictest24105-21grep09:04:421
1765599743,49cyclictest357-21runrttasks07:12:291
1765599742,67cyclictest0-21swapper/110:04:211
17648997454,14cyclictest10315-21latency_hist09:39:080
1764899743,38cyclictest2633-21netstat10:19:320
1764899743,38cyclictest2633-21netstat10:19:320
17648997413,36cyclictest161ktimers/012:29:210
17661997357,10cyclictest2797-21sendmail_mailqu10:19:382
17661997357,10cyclictest2797-21sendmail_mailqu10:19:382
17661997354,14cyclictest32156-21switchtime07:49:422
1766199734,27cyclictest17174-21df_inode10:44:232
17655997344,20cyclictest24960-21cut07:29:281
1765599733,60cyclictest16735-21apt-get08:44:151
17655997326,40cyclictest0-21swapper/110:39:351
17648997346,10cyclictest181rcu_preempt07:34:340
1764899734,53cyclictest25841-21fschecks_count10:04:270
17648997332,11cyclictest181rcu_preempt11:41:390
17648997326,13cyclictest0-21swapper/011:37:080
1756827311,41sleep117586-21unin-run07:09:071
1766999723,31cyclictest14624-21chrt11:34:073
1766999723,30cyclictest31970-21chrt11:09:063
17669997213,39cyclictest0-21swapper/309:30:113
17669997213,39cyclictest0-21swapper/309:30:113
1766199726,49cyclictest11623-21latency_hist08:29:072
17661997256,10cyclictest26453-21/usr/sbin/munin10:59:282
17661997255,11cyclictest357-21runrttasks09:38:432
17661997252,11cyclictest23193-21cpuspeed07:24:202
17661997251,12cyclictest181rcu_preempt08:39:172
17661997249,13cyclictest0-21swapper/208:34:272
17661997244,20cyclictest18136-21fschecks_time11:39:262
17661997237,19cyclictest10734-21kworker/u17:0-ext4-rsv-conversion10:49:252
1766199723,34cyclictest1926-21latency09:24:292
17661997231,9cyclictest0-21swapper/210:11:232
17655997231,33cyclictest0-21swapper/108:24:411
17655997226,39cyclictest30119-21cut07:44:571
1764899728,41cyclictest15-21ksoftirqd/011:14:170
17648997256,9cyclictest14867-21ssh11:34:100
1764899723,50cyclictest3550-21modprobe08:04:050
1764899723,43cyclictest15-21ksoftirqd/007:54:230
1764899722,37cyclictest15-21ksoftirqd/007:34:060
17648997213,16cyclictest15-21ksoftirqd/008:42:150
162012726,45sleep30-21swapper/307:04:203
17661997155,11cyclictest17638-21cyclictest09:54:272
17661997153,12cyclictest293-21systemd-logind08:59:062
17661997153,12cyclictest293-21systemd-logind08:59:062
17661997119,24cyclictest24143-21grep09:04:422
1765599713,58cyclictest26504-21diskstats07:34:261
17655997123,41cyclictest5362-21sed08:09:091
1764899716,33cyclictest14982-21latency_hist12:29:070
1764899712,41cyclictest15-21ksoftirqd/009:09:400
266202700,5sleep20-21swapper/209:11:302
1766999703,26cyclictest27688-21taskset11:01:013
1766999702,30cyclictest27067-21chrt07:34:403
17655997029,34cyclictest0-21swapper/111:19:191
1764899703,34cyclictest673-21latency12:04:310
1764899703,34cyclictest673-21latency12:04:310
17648997025,32cyclictest24550-21diskmemload09:54:310
1766999693,25cyclictest31385-21taskset07:49:103
1766999692,28cyclictest26635-21chrt11:54:093
1766999692,28cyclictest26635-21chrt11:54:093
1766999692,28cyclictest15427-21taskset08:39:273
1766999692,28cyclictest14255-21chrt10:39:273
1766999692,27cyclictest24325-21chrt11:49:353
17661996951,12cyclictest24550-21diskmemload12:09:272
17661996949,14cyclictest19915-21latency_hist08:54:092
1766199693,52cyclictest7723-21awk10:29:052
1766199693,47cyclictest2796-21ps07:59:372
17661996934,12cyclictest181rcu_preempt09:51:412
17661996934,12cyclictest181rcu_preempt09:51:412
17661996930,16cyclictest181rcu_preempt10:40:012
17655996950,12cyclictest0-21swapper/110:59:381
1766999682,31cyclictest4856-21chrt11:17:333
17661996846,14cyclictest17638-21cyclictest08:44:252
17661996845,16cyclictest10695-21latency_hist10:34:072
17661996839,10cyclictest17638-21cyclictest07:59:082
1766199682,28cyclictest11392-21munin-run11:29:072
1766999675,25cyclictest26507-21chrt10:04:423
1766199678,22cyclictest216882cstates08:59:222
1766199673,40cyclictest21011-21fschecks_count11:44:262
17661996729,10cyclictest19308-21ssh12:34:482
17661996729,10cyclictest19308-21ssh12:34:482
1766999668,38cyclictest0-21swapper/311:34:323
1766999662,24cyclictest10202-21chrt08:24:083
17669996620,23cyclictest0-21swapper/312:19:243
17661996643,15cyclictest29700-21sh11:59:212
1766199663,43cyclictest29729-21latency_hist07:44:102
1766199663,42cyclictest5831-21taskset08:09:312
1766199662,45cyclictest357-21runrttasks08:32:402
1766999653,26cyclictest24729-21chrt07:29:173
1766999653,24cyclictest6157-21chrt12:14:203
1766999653,24cyclictest18577-21taskset12:34:303
1766999653,24cyclictest18577-21taskset12:34:303
1766999653,23cyclictest32369-21chrt12:04:063
1766999653,22cyclictest6117-21taskset08:09:393
1766999653,21cyclictest17871-21chrt07:09:213
1766999652,20cyclictest1554-21chrt12:05:283
1766999652,20cyclictest1554-21chrt12:05:283
17661996542,14cyclictest0-21swapper/207:29:272
1766999643,20cyclictest7382-21chrt08:14:293
1766999643,20cyclictest7382-21chrt08:14:293
1766999642,28cyclictest29915-21taskset10:10:253
1766999642,27cyclictest31111-21chrt09:19:253
17661996426,11cyclictest181rcu_preempt09:45:442
1766199642,43cyclictest3831-21apt-get08:04:172
1766999633,24cyclictest5764-21chrt11:19:193
1766999633,24cyclictest30334-21chrt09:18:283
1766999633,22cyclictest4151-21chrt10:22:253
1766999633,22cyclictest4151-21chrt10:22:253
1766999632,27cyclictest6296-21chrt10:25:043
1766999632,26cyclictest24680-21chrt07:29:093
1766999632,25cyclictest10493-21chrt10:33:313
1766999632,24cyclictest28959-21chrt07:39:423
1766999632,22cyclictest31583-21chrt07:49:263
1766199634,32cyclictest21509-21/usr/sbin/munin07:19:222
1766999623,22cyclictest11899-21chrt08:29:193
1766999623,22cyclictest11891-21chrt09:40:053
1766999623,22cyclictest11891-21chrt09:40:053
1766999623,20cyclictest19681-21chrt09:54:203
1766999622,24cyclictest20240-21taskset11:43:273
1766999622,23cyclictest17132-21chrt10:44:213
1766999622,22cyclictest18450-21chrt09:51:483
1766999622,22cyclictest18450-21chrt09:51:483
1766999622,21cyclictest1715-21chrt11:12:023
1766999622,21cyclictest1715-21chrt11:12:023
1766999622,21cyclictest12445-21chrt10:35:383
1766999613,23cyclictest3196-21taskset07:59:433
1766999613,22cyclictest14593-21chrt08:36:073
1766999613,20cyclictest14807-21chrt09:44:553
1766999613,20cyclictest10417-21chrt09:39:083
1766999612,27cyclictest22871-21taskset09:03:253
1766999612,25cyclictest26040-21chrt09:09:533
1766999612,24cyclictest1740-21chrt07:56:553
1766999612,22cyclictest5092-21chrt12:12:093
1766999612,22cyclictest19987-21chrt10:49:163
1766999612,21cyclictest1456-21chrt09:24:163
1766999612,20cyclictest11319-21chrt11:29:043
1766999606,13cyclictest0-21swapper/312:29:293
1766999603,22cyclictest22613-21taskset11:46:473
1766999602,21cyclictest20648-21chrt08:54:383
1766999602,21cyclictest20648-21chrt08:54:383
1766999602,21cyclictest14640-21taskset12:28:223
1766999602,20cyclictest22059-21chrt07:19:333
1766999602,20cyclictest17827-21chrt08:45:553
1766999602,20cyclictest11420-21chrt08:26:353
1766999594,12cyclictest318272sleep310:14:263
1766999593,22cyclictest20929-21chrt07:15:243
1766999593,22cyclictest20929-21chrt07:15:243
1766999593,21cyclictest19359-21chrt08:49:563
1766999583,19cyclictest24304-21chrt09:06:083
1766999582,22cyclictest4919-21taskset08:06:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional