You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 05:31
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Mon May 25, 2026 00:44:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2289721389,96sleep122950-21grep18:56:161
23742991212,88cyclictest22163-21taskset00:20:550
237749911768,42cyclictest9478-21sh23:06:022
2347621176,16sleep20-21swapper/219:00:162
23742991133,74cyclictest418-21fschecks_time19:25:520
237429911242,49cyclictest352-21runrttasks21:04:080
237429911242,49cyclictest352-21runrttasks21:04:080
237429911118,52cyclictest0-21swapper/022:40:470
237429911118,52cyclictest0-21swapper/022:40:470
23742991093,75cyclictest14979-21ps21:26:230
23742991093,75cyclictest14979-21ps21:26:230
23742991083,71cyclictest0-21swapper/022:25:310
237429910812,52cyclictest0-21swapper/022:45:570
23742991073,63cyclictest0-21swapper/000:25:530
23742991073,63cyclictest0-21swapper/000:25:530
23742991062,68cyclictest31082-21diskmemload21:40:450
23742991062,67cyclictest15437-21awk20:10:550
237749910580,15cyclictest15801-21sh00:10:542
237749910540,12cyclictest0-21swapper/221:55:522
23742991054,83cyclictest24063-21apt-get19:01:210
23742991052,70cyclictest0-21swapper/023:16:030
23742991052,70cyclictest0-21swapper/023:16:030
237429910442,55cyclictest0-21swapper/023:25:530
237429910442,55cyclictest0-21swapper/023:25:530
237429910329,44cyclictest2541-21kworker/u17:3-rpciod00:01:050
23774991022,41cyclictest29834-21runrttasks23:40:392
237429910218,51cyclictest18161-21/usr/sbin/munin20:20:520
237429910184,11cyclictest17947-21systemd-run20:20:300
237749910073,18cyclictest3048-21gpgv23:50:342
237749910073,18cyclictest3048-21gpgv23:50:342
237429910038,52cyclictest0-21swapper/023:46:000
23742991003,62cyclictest0-21swapper/022:20:290
23742991003,58cyclictest0-21swapper/021:35:440
23742991002,68cyclictest0-21swapper/019:10:500
23742991002,66cyclictest6732-21df23:55:500
23742991002,63cyclictest8287-21/usr/sbin/munin21:15:520
23742991002,62cyclictest30092-21grep21:51:060
23742991002,62cyclictest30092-21grep21:51:060
2374299997,57cyclictest0-21swapper/019:32:070
2374299997,57cyclictest0-21swapper/019:32:070
2374299995,49cyclictest0-21swapper/023:33:050
2374299995,49cyclictest0-21swapper/023:33:050
2374299993,59cyclictest21106-21ssh00:19:520
2374299992,60cyclictest31289-21tune2fs19:20:510
2374299992,59cyclictest17079-21kworker/u17:1+rpciod21:32:040
23742999915,49cyclictest94912chrt19:51:130
2374299984,59cyclictest0-21swapper/021:25:300
2374299984,59cyclictest0-21swapper/021:25:300
2374299983,90cyclictest0-21swapper/023:05:440
23742999836,52cyclictest0-21swapper/022:40:160
2374299983,61cyclictest12371-21ntpq23:10:590
2374299982,61cyclictest0-21swapper/000:11:010
2374299982,60cyclictest5056-21seq22:05:090
2374299982,58cyclictest24999-21expr20:40:470
23774999750,10cyclictest11906-21timerwakeupswit21:21:052
23774999750,10cyclictest11906-21timerwakeupswit21:21:052
2377499973,38cyclictest8592-21cpuspeed22:10:452
2375399978,67cyclictest14171-21apt-get21:25:511
2375399978,67cyclictest14171-21apt-get21:25:511
23753999769,19cyclictest7092-21kworker/u17:0+events_unbound23:05:281
23753999769,19cyclictest7092-21kworker/u17:0+events_unbound23:05:281
23742999737,52cyclictest0-21swapper/022:56:020
2374299972,65cyclictest19367-21taskset23:22:420
2374299972,60cyclictest5222-21modprobe19:40:320
2374299972,60cyclictest5222-21modprobe19:40:320
2374299972,60cyclictest2197-21cron22:00:280
2374299972,60cyclictest12996-21seq20:04:530
2374299972,60cyclictest12996-21seq20:04:530
2374299963,79cyclictest8504-21apt-get22:10:430
2374299963,60cyclictest8170-21munin-run22:10:280
2374299963,56cyclictest19944-21diskstats20:25:500
2374299962,66cyclictest352-21runrttasks21:12:510
23742999618,50cyclictest5074-21latency_hist19:40:310
23742999618,50cyclictest5074-21latency_hist19:40:310
23742999614,54cyclictest26024-21acpi21:45:390
2374299953,85cyclictest26877-21runrttasks23:35:410
2374299953,66cyclictest31273-21latency_hist21:00:330
2374299953,61cyclictest29909-21awk19:15:570
2374299953,58cyclictest23081-21grep20:35:310
2374299952,61cyclictest0-21swapper/022:30:580
2374299952,57cyclictest4663-21cat23:52:000
2374299952,57cyclictest4663-21cat23:52:000
23742999510,55cyclictest32671-21idleruntime22:50:550
2374299943,55cyclictest0-21swapper/022:30:310
2374299938,50cyclictest0-21swapper/021:05:530
2374299932,75cyclictest26487-21apt-get20:45:400
2374299932,61cyclictest29798-21ssh23:40:370
2374299913,56cyclictest15246-21munin-run00:10:310
2374299913,54cyclictest0-21swapper/019:49:290
23774999062,18cyclictest3059-21sh22:55:492
2374299904,57cyclictest0-21swapper/023:01:500
2374299904,57cyclictest0-21swapper/023:01:500
23742998933,49cyclictest0-21swapper/020:50:520
23753998865,17cyclictest352-21runrttasks22:50:531
23753998765,15cyclictest8684-21sh22:10:481
2374299873,74cyclictest0-21swapper/020:35:410
2374299873,74cyclictest0-21swapper/020:35:410
2374299872,56cyclictest0-21swapper/020:05:550
2377499862,47cyclictest30741-21sh19:20:292
23753998628,27cyclictest31-21ksoftirqd/120:56:421
23774998568,10cyclictest23555-21sh23:30:312
23774998568,10cyclictest23555-21sh23:30:312
2375399853,38cyclictest31-21ksoftirqd/123:16:301
2375399853,38cyclictest31-21ksoftirqd/123:16:301
23753998517,27cyclictest31-21ksoftirqd/121:10:561
2374299853,72cyclictest23761-21cmp19:00:340
23774998442,11cyclictest181rcu_preempt23:24:592
2374299842,73cyclictest27007-21timerwakeupswit19:06:080
2374299842,73cyclictest27007-21timerwakeupswit19:06:080
23774998362,15cyclictest8278-21sh23:57:572
23742998324,52cyclictest0-21swapper/019:55:510
23774998257,16cyclictest13420-21sh23:12:422
23774998247,9cyclictest181rcu_preempt23:50:292
23774998241,11cyclictest181rcu_preempt19:55:322
2375399826,32cyclictest31-21ksoftirqd/121:58:311
2375399822,58cyclictest15155-21apt-get00:10:531
2218228226,29sleep30-21swapper/318:55:493
2377499813,34cyclictest10438-21timerwakeupswit00:01:052
2377499812,57cyclictest32353-21munin-plugin-st19:25:292
2375399816,44cyclictest31-21ksoftirqd/120:19:231
2375399812,45cyclictest31-21ksoftirqd/119:21:031
23753998116,26cyclictest31-21ksoftirqd/121:35:471
23753998112,34cyclictest31-21ksoftirqd/119:57:451
2379099802,36cyclictest20331-21chrt00:17:403
2377499808,12cyclictest0-21swapper/221:45:412
2375399806,41cyclictest31-21ksoftirqd/123:32:531
2375399806,41cyclictest31-21ksoftirqd/123:32:531
23753998056,16cyclictest352-21runrttasks00:00:501
2375399803,68cyclictest3197-21apt-config23:50:321
2379099794,36cyclictest31851-21chrt23:43:423
2375399793,49cyclictest31-21ksoftirqd/119:20:281
23753997925,29cyclictest31-21ksoftirqd/122:33:011
2375399792,50cyclictest31-21ksoftirqd/119:41:061
2375399792,50cyclictest31-21ksoftirqd/119:41:061
23753997917,31cyclictest31-21ksoftirqd/122:49:521
23774997857,15cyclictest3448-21latency_hist19:35:312
23774997857,15cyclictest3448-21latency_hist19:35:312
2377499783,36cyclictest13564-21grep00:06:072
2377499782,50cyclictest32600-21/usr/sbin/munin19:25:462
2375399785,53cyclictest31-21ksoftirqd/123:06:461
2375399785,34cyclictest31-21ksoftirqd/120:05:471
23753997826,23cyclictest31-21ksoftirqd/120:54:431
23753997814,33cyclictest31-21ksoftirqd/121:09:411
23774997757,14cyclictest3048-21sh22:00:562
23774997757,14cyclictest1653-21systemd-run21:05:302
23774997757,14cyclictest1653-21systemd-run21:05:302
2375399777,30cyclictest31-21ksoftirqd/119:26:461
23753997751,14cyclictest0-21swapper/100:20:281
23753997744,9cyclictest181rcu_preempt00:10:331
2375399774,30cyclictest31-21ksoftirqd/119:38:451
2375399774,30cyclictest31-21ksoftirqd/119:38:451
2375399773,46cyclictest3940-21grep23:51:421
2375399773,46cyclictest3940-21grep23:51:421
2375399772,35cyclictest31-21ksoftirqd/119:04:311
23790997615,44cyclictest0-21swapper/300:10:473
23753997659,11cyclictest23791-21ntp_offset20:35:581
23753997659,11cyclictest23791-21ntp_offset20:35:581
2375399762,45cyclictest31-21ksoftirqd/120:46:371
2375399762,42cyclictest31-21ksoftirqd/123:44:081
2375399762,40cyclictest31-21ksoftirqd/122:17:251
23753997618,29cyclictest31-21ksoftirqd/120:20:511
23753997610,45cyclictest31082-21diskmemload21:33:581
23774997557,13cyclictest23722-21cyclictest19:45:462
23774997538,11cyclictest31098-21ssh21:53:482
23774997538,11cyclictest31098-21ssh21:53:482
23753997558,11cyclictest28681-21ssh21:50:291
23753997544,7cyclictest181rcu_preempt21:54:111
23753997544,7cyclictest181rcu_preempt21:54:111
2375399752,40cyclictest31-21ksoftirqd/120:01:381
2375399752,40cyclictest31-21ksoftirqd/120:01:381
23753997520,34cyclictest31-21ksoftirqd/119:13:451
2375399751,42cyclictest31-21ksoftirqd/120:32:151
23774997451,16cyclictest17344-21sh21:30:502
23774997417,14cyclictest181rcu_preempt22:40:522
23774997417,14cyclictest181rcu_preempt22:40:522
2375399749,27cyclictest31-21ksoftirqd/119:48:431
23753997431,12cyclictest181rcu_preempt22:27:231
152082742,21sleep115269-21sed22:21:041
2377499734,48cyclictest25500-21grep20:41:022
2377499733,56cyclictest27502-21apt19:10:282
2377499733,56cyclictest27502-21apt19:10:282
23774997324,15cyclictest181rcu_preempt19:05:292
23774997311,33cyclictest28053-21fschecks_time19:10:512
2375399735,41cyclictest31-21ksoftirqd/123:11:451
23753997354,11cyclictest0-21swapper/122:06:081
23753997348,16cyclictest26271-21latency_hist20:45:301
23753997345,20cyclictest11347-21sh21:20:541
23753997345,20cyclictest11347-21sh21:20:541
23753997333,13cyclictest181rcu_preempt22:59:271
2375399732,31cyclictest31-21ksoftirqd/119:10:321
2375399732,31cyclictest31-21ksoftirqd/119:10:321
2375399731,40cyclictest31-21ksoftirqd/122:36:071
23774997254,10cyclictest0-21swapper/222:51:082
23774997252,11cyclictest0-21swapper/200:25:532
23774997252,11cyclictest0-21swapper/200:25:532
2377499723,61cyclictest352-21runrttasks21:25:512
2377499723,61cyclictest352-21runrttasks21:25:512
2377499723,37cyclictest21654-21apt-get20:30:422
2377499723,24cyclictest22785-21ssh22:34:452
23753997243,8cyclictest181rcu_preempt19:55:081
2375399723,42cyclictest31-21ksoftirqd/123:56:091
23753997232,8cyclictest181rcu_preempt22:01:351
23753997227,11cyclictest181rcu_preempt23:27:541
23753997227,11cyclictest181rcu_preempt23:27:541
2377499716,55cyclictest23332-21apt-get23:30:392
2377499716,55cyclictest23332-21apt-get23:30:392
2377499716,36cyclictest10616-21modprobe21:20:282
23774997150,15cyclictest26710-21fschecks_time20:45:502
2377499714,33cyclictest13003-21sh22:17:172
2377499713,30cyclictest10088-21/usr/sbin/munin19:56:022
2375399718,38cyclictest31-21ksoftirqd/122:41:491
2375399718,38cyclictest31-21ksoftirqd/122:41:491
2375399712,36cyclictest31-21ksoftirqd/120:26:571
2377499702,27cyclictest5877-21awk19:41:032
2377499702,27cyclictest5877-21awk19:41:032
23753997054,10cyclictest21649-21diskmemload00:20:391
23753997051,14cyclictest23722-21cyclictest21:01:041
23753997051,14cyclictest23722-21cyclictest21:01:041
23774996952,11cyclictest18698-21ntp_offset20:20:572
23774996949,14cyclictest24017-21ntp_offset22:36:042
23774996948,12cyclictest0-21swapper/222:06:032
23753996936,16cyclictest181rcu_preempt23:21:191
23753996926,27cyclictest31-21ksoftirqd/121:19:221
23753996913,36cyclictest31-21ksoftirqd/121:42:181
23753996911,25cyclictest31-21ksoftirqd/100:27:461
23753996911,25cyclictest31-21ksoftirqd/100:27:461
23774996853,10cyclictest39-21ksoftirqd/221:40:512
2377499684,44cyclictest22857-21ssh21:40:322
2377499684,28cyclictest29958-21/usr/sbin/munin22:46:002
23753996829,8cyclictest181rcu_preempt23:35:491
2375399682,38cyclictest31-21ksoftirqd/120:11:541
23774996747,12cyclictest0-21swapper/220:15:512
23774996741,17cyclictest352-21runrttasks20:00:562
23774996741,17cyclictest352-21runrttasks20:00:562
23774996733,13cyclictest181rcu_preempt20:06:122
23774996730,11cyclictest39-21ksoftirqd/200:16:082
23774996730,10cyclictest181rcu_preempt20:10:402
23774996728,10cyclictest181rcu_preempt23:18:342
23774996728,10cyclictest181rcu_preempt23:18:342
2377499672,29cyclictest3779-21ssh21:08:102
23774996714,36cyclictest5327-21cat21:10:522
23774996714,12cyclictest181rcu_preempt20:26:062
2375399669,29cyclictest31-21ksoftirqd/119:31:131
2375399669,29cyclictest31-21ksoftirqd/119:31:131
23774996548,11cyclictest23722-21cyclictest19:35:522
23774996548,11cyclictest23722-21cyclictest19:35:522
23774996547,13cyclictest15239-21sensors_temp22:21:032
23774996544,15cyclictest18056-21sh22:26:012
2377499652,32cyclictest29261-21ssh23:39:472
2377499651,46cyclictest12077-1kworker/2:2H+kblockd00:21:582
2377499643,42cyclictest7036-21runrttasks23:01:292
2377499643,42cyclictest7036-21runrttasks23:01:292
2379099633,44cyclictest917-21kworker/3:1+events21:35:463
2377499634,44cyclictest24617-21munin-plugin-st20:40:292
2377499634,44cyclictest24617-21munin-plugin-st20:40:292
2377499632,24cyclictest28631-21ntpq20:50:572
23774996212,32cyclictest31085-21munin-plugin-st21:00:292
2379099593,36cyclictest917-21kworker/3:1+events21:01:073
2379099593,36cyclictest917-21kworker/3:1+events21:01:073
2379099592,39cyclictest917-21kworker/3:1+events20:50:553
2379099583,44cyclictest3931-21taskset23:51:413
2379099583,44cyclictest3931-21taskset23:51:413
2379099572,41cyclictest15859-21chrt20:11:043
23790995722,24cyclictest24566-21chrt23:31:593
23790995722,24cyclictest24566-21chrt23:31:593
2379099565,41cyclictest0-21swapper/321:25:503
2379099565,41cyclictest0-21swapper/321:25:503
23790995642,8cyclictest0-21swapper/323:50:313
2379099563,39cyclictest10022-21taskset23:06:333
2379099563,38cyclictest7727-21chrt19:46:053
2379099563,37cyclictest23328-21taskset21:40:513
2379099562,40cyclictest7653-21taskset23:56:123
2379099562,40cyclictest2042-21chrt19:30:463
2379099562,40cyclictest2042-21chrt19:30:463
2379099554,35cyclictest16778-21chrt20:15:503
2379099553,36cyclictest26254-21taskset21:45:503
2379099553,34cyclictest18494-21taskset23:21:043
2379099552,38cyclictest20011-21taskset21:35:323
2379099552,35cyclictest8585-21chrt23:05:323
2379099552,35cyclictest8585-21chrt23:05:323
2379099549,33cyclictest31107-21chrt21:00:293
2379099543,38cyclictest7598-21chrt21:15:253
2379099543,36cyclictest9347-21chrt19:51:043
23790995425,21cyclictest320002chrt21:55:393
2379099542,40cyclictest14346-21taskset22:20:323
2379099542,38cyclictest24173-21chrt19:01:263
2379099537,37cyclictest24113-21chrt00:24:573
2379099533,35cyclictest4183-21chrt21:09:173
2379099533,34cyclictest21178-21chrt22:31:023
2379099533,34cyclictest10118-21chrt22:12:333
2379099533,32cyclictest31115-21chrt19:20:463
2379099532,37cyclictest950-21chrt19:26:053
2379099532,37cyclictest31938-21chrt22:50:283
2379099523,34cyclictest17645-21chrt22:25:513
2379099523,33cyclictest15477-21chrt23:16:023
2379099523,33cyclictest15477-21chrt23:16:023
2379099523,32cyclictest8156-21chrt21:15:503
2379099522,39cyclictest12520-21taskset21:22:273
2379099522,39cyclictest12520-21taskset21:22:273
2379099522,36cyclictest20416-21chrt20:26:033
2379099522,35cyclictest11754-21chrt23:10:383
2379099522,32cyclictest23567-21chrt22:35:523
2379099522,32cyclictest10261-21chrt19:55:493
2379099514,33cyclictest23734-21chrt23:30:333
2379099514,33cyclictest23734-21chrt23:30:333
2379099513,35cyclictest3374-21taskset22:55:553
2379099513,34cyclictest28169-21taskset22:43:103
2379099513,34cyclictest28169-21taskset22:43:103
2379099513,33cyclictest9725-21taskset00:00:503
2379099513,33cyclictest27068-21chrt19:06:083
2379099513,33cyclictest27068-21chrt19:06:083
2379099512,38cyclictest1353-21chrt22:52:143
2379099512,36cyclictest13069-21chrt20:05:293
2379099512,36cyclictest13069-21chrt20:05:293
2379099512,35cyclictest3249-21chrt22:01:023
2379099512,31cyclictest3697-21taskset19:35:453
2379099512,31cyclictest3697-21taskset19:35:453
2379099504,36cyclictest0-21swapper/320:30:423
2379099504,34cyclictest27908-21chrt19:10:463
2379099503,35cyclictest14304-21chrt20:06:063
2379099503,34cyclictest26680-21chrt00:28:503
2379099503,34cyclictest26680-21chrt00:28:503
2379099503,33cyclictest23479-21chrt20:35:513
2379099503,33cyclictest23479-21chrt20:35:513
2379099503,31cyclictest5339-21chrt19:40:473
2379099503,31cyclictest5339-21chrt19:40:473
2379099494,34cyclictest26313-21taskset20:45:313
2379099493,31cyclictest29066-21chrt21:50:443
2379099493,31cyclictest29066-21chrt21:50:443
2379099492,33cyclictest28388-21chrt23:37:183
2379099483,31cyclictest18501-21chrt20:20:523
2379099482,34cyclictest6022-21chrt22:05:573
2379099472,33cyclictest14614-21chrt00:09:003
2379099465,29cyclictest27026-21chrt20:45:593
2379099453,26cyclictest29555-21chrt19:15:483
2379099452,31cyclictest16293-21chrt22:23:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional