You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 00:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Wed Apr 08, 2026 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17865991183,31cyclictest24375-21grep10:49:110
17865991183,31cyclictest24375-21grep10:49:110
1746321096,16sleep20-21swapper/207:00:162
178659910639,11cyclictest0-21swapper/007:53:360
17865991043,31cyclictest24525-21timerwakeupswit07:20:110
17865991043,31cyclictest24525-21timerwakeupswit07:20:110
672421010,6sleep30-21swapper/311:14:223
672421010,6sleep30-21swapper/311:14:223
17880999947,12cyclictest181rcu_preempt07:48:472
1786599983,24cyclictest7104-21grep11:14:320
169362989,63sleep116935-21ntp_states06:59:131
17880999455,13cyclictest181rcu_preempt11:44:032
17880999455,13cyclictest181rcu_preempt11:44:032
17880999443,11cyclictest181rcu_preempt11:53:492
17880999443,11cyclictest181rcu_preempt11:53:492
17872999377,9cyclictest10890-21munin-run09:33:351
17865999166,17cyclictest17858-21cyclictest07:53:520
17865999048,35cyclictest0-21swapper/009:14:110
17880998962,21cyclictest9572-21unixbench_singl10:24:132
1788899889,66cyclictest26644-21chrt10:53:543
17880998847,10cyclictest181rcu_preempt08:29:082
17872998872,10cyclictest26703-21ssh10:54:041
17872998869,13cyclictest15056-21munin-plugin-st08:33:351
17865998768,13cyclictest23632-21fschecks_time08:58:570
17865998755,10cyclictest181rcu_preempt09:03:510
17880998657,14cyclictest9540-21ssh09:29:502
17880998549,8cyclictest181rcu_preempt07:58:372
17880998454,9cyclictest17809-21ssh09:44:092
17880998452,22cyclictest608-21sh11:04:042
17880998452,22cyclictest608-21sh11:04:042
17880998449,9cyclictest3187-21ssh12:02:082
17880998439,12cyclictest181rcu_preempt10:29:082
17880998439,12cyclictest181rcu_preempt10:29:082
1787299843,46cyclictest31-21ksoftirqd/111:11:091
1787299843,46cyclictest31-21ksoftirqd/111:11:091
1787299842,37cyclictest31-21ksoftirqd/112:08:311
17865998460,15cyclictest10781-21ssh11:20:240
185462830,9sleep00-21swapper/008:43:450
17880998351,14cyclictest3350-21ssh09:19:462
17872998369,9cyclictest9717-21ssh12:13:361
17872998369,9cyclictest9717-21ssh12:13:361
1787299833,44cyclictest31-21ksoftirqd/112:15:061
17872998310,48cyclictest31-21ksoftirqd/110:21:021
17865998364,13cyclictest24757-21diskmemload10:39:070
17865998364,13cyclictest24757-21diskmemload10:39:070
17865998350,26cyclictest21116-21ssh11:38:380
17865998350,26cyclictest21116-21ssh11:38:380
1639628311,48sleep016398-21apt-get06:58:470
1787299826,41cyclictest31-21ksoftirqd/107:20:091
1787299826,41cyclictest31-21ksoftirqd/107:20:091
17872998258,18cyclictest30561-21sh11:53:501
17872998258,18cyclictest30561-21sh11:53:501
17872998213,45cyclictest31-21ksoftirqd/107:15:511
17865998262,14cyclictest17819-21proc_pri09:44:090
17865998243,9cyclictest181rcu_preempt12:23:470
17880998162,13cyclictest24757-21diskmemload12:19:042
17880998162,13cyclictest24757-21diskmemload11:25:012
17880998162,13cyclictest24757-21diskmemload11:25:012
17880998152,8cyclictest5603-21rm11:12:382
17880998152,8cyclictest5603-21rm11:12:382
17872998164,11cyclictest300-21rs:main1
17872998146,11cyclictest181rcu_preempt11:23:361
1787299813,45cyclictest31-21ksoftirqd/109:48:351
17880998046,11cyclictest181rcu_preempt11:17:182
17880998044,13cyclictest181rcu_preempt11:48:492
17872998064,10cyclictest15969-21ssh09:41:111
17865998043,10cyclictest181rcu_preempt10:43:450
17880997948,8cyclictest181rcu_preempt10:49:452
17880997948,8cyclictest181rcu_preempt10:49:452
17880997945,13cyclictest18059-21ssh10:39:062
17880997945,13cyclictest18059-21ssh10:39:062
17865997965,9cyclictest15-21ksoftirqd/009:23:190
1786599793,38cyclictest27765-21apt-get09:08:520
1786599793,38cyclictest27765-21apt-get09:08:520
17880997864,9cyclictest381ktimers/210:18:482
1787299782,25cyclictest31-21ksoftirqd/110:27:241
17865997854,15cyclictest14188-21ssh12:19:120
1786599783,52cyclictest21884-21modprobe12:33:350
1786599783,52cyclictest21884-21modprobe12:33:350
17880997756,15cyclictest15535-21unixbench_singl10:34:122
17872997761,10cyclictest29212-21cat10:03:521
17872997757,13cyclictest24392-21apt-key11:43:391
17865997757,14cyclictest25378-21kworker/u17:1+xprtiod11:14:050
17865997757,14cyclictest25378-21kworker/u17:1+xprtiod11:14:050
17880997646,9cyclictest181rcu_preempt12:08:352
17880997639,8cyclictest181rcu_preempt12:24:062
17880997635,13cyclictest181rcu_preempt11:39:422
17872997661,9cyclictest6451-21timerandwakeup08:04:101
1787299764,40cyclictest31-21ksoftirqd/109:58:571
17872997628,12cyclictest181rcu_preempt11:48:431
17865997651,17cyclictest14487-21fschecks_time09:38:590
17865997649,19cyclictest21865-21netstat11:39:020
17880997552,17cyclictest27758-21latency_hist09:08:412
17880997552,17cyclictest27758-21latency_hist09:08:412
17880997546,12cyclictest181rcu_preempt07:03:422
17872997551,17cyclictest16862-21sh12:24:081
1787299753,27cyclictest31-21ksoftirqd/110:40:001
1787299753,27cyclictest31-21ksoftirqd/110:40:001
1787299752,36cyclictest31-21ksoftirqd/112:23:151
17865997552,15cyclictest24757-21diskmemload11:31:170
17865997551,17cyclictest741-21latency_hist11:58:390
17865997551,17cyclictest741-21latency_hist11:58:390
17865997548,16cyclictest30058-21ssh11:53:170
1786599754,38cyclictest2857-21fschecks_time10:13:580
17872997439,8cyclictest181rcu_preempt10:49:021
17872997439,8cyclictest181rcu_preempt10:49:021
1787299742,30cyclictest31-21ksoftirqd/110:33:511
17865997456,13cyclictest30783-21timerandwakeup07:39:100
17865997445,9cyclictest181rcu_preempt08:28:390
17865997431,8cyclictest181rcu_preempt08:19:040
17880997344,21cyclictest16681-21sh11:29:532
17880997342,8cyclictest181rcu_preempt08:49:082
17880997341,8cyclictest27369-21ssh09:07:322
17880997340,7cyclictest181rcu_preempt12:09:032
17880997340,7cyclictest181rcu_preempt12:09:032
17880997321,16cyclictest181rcu_preempt10:55:172
17872997357,10cyclictest17858-21cyclictest09:53:451
17872997356,11cyclictest8984-21systemd-run11:18:361
17872997344,21cyclictest29769-21cut07:38:381
17872997330,31cyclictest11819-21modprobe08:23:351
1787299732,32cyclictest31-21ksoftirqd/109:15:301
1787299732,24cyclictest31-21ksoftirqd/109:10:491
1787299732,24cyclictest31-21ksoftirqd/109:10:491
1787299731,37cyclictest31-21ksoftirqd/108:59:071
17872997312,9cyclictest301ktimers/111:28:571
17865997354,13cyclictest15-21ksoftirqd/009:33:390
17865997348,19cyclictest16164-21timerwakeupswit08:34:110
17865997325,38cyclictest11461-21sort10:28:460
17865997325,38cyclictest11461-21sort10:28:460
17880997251,13cyclictest10293-21sh12:13:562
17880997239,11cyclictest181rcu_preempt08:08:382
17880997238,11cyclictest181rcu_preempt07:23:502
17880997236,13cyclictest181rcu_preempt09:52:292
17880997236,13cyclictest181rcu_preempt09:52:292
17880997236,13cyclictest181rcu_preempt08:59:152
17880997234,8cyclictest181rcu_preempt10:43:582
17880997234,10cyclictest181rcu_preempt09:13:472
17880997231,9cyclictest181rcu_preempt09:37:282
17872997252,13cyclictest0-21swapper/109:49:011
17872997252,13cyclictest0-21swapper/109:49:011
17865997255,11cyclictest17858-21cyclictest12:06:220
17865997249,15cyclictest695-21ntp_offset11:04:050
17865997249,15cyclictest695-21ntp_offset11:04:050
17865997243,13cyclictest181rcu_preempt09:33:520
17880997133,8cyclictest181rcu_preempt11:19:132
1788099713,23cyclictest32622-21/usr/sbin/munin10:09:042
1788099713,23cyclictest32622-21/usr/sbin/munin10:09:042
1787299717,44cyclictest31-21ksoftirqd/108:25:501
17872997149,15cyclictest27418-21mailstats07:29:071
17872997148,7cyclictest181rcu_preempt10:15:051
1787299712,31cyclictest31-21ksoftirqd/107:25:251
1787299712,24cyclictest31-21ksoftirqd/108:53:561
1787299712,24cyclictest31-21ksoftirqd/108:53:561
1787299712,24cyclictest19440-21ssh11:34:131
1787299712,24cyclictest19440-21ssh11:34:131
1787299711,47cyclictest31-21ksoftirqd/110:48:331
17872997111,43cyclictest24530-21apt-get11:43:501
17872997111,43cyclictest24530-21apt-get11:43:501
1786599719,49cyclictest12533-21sh11:24:050
1786599719,49cyclictest12533-21sh11:24:050
17865997136,12cyclictest30099-21unixbench_singl10:04:120
17865997135,15cyclictest181rcu_preempt08:12:480
17865997135,15cyclictest181rcu_preempt08:12:480
17865997112,28cyclictest20198-21cpu09:48:520
17865997112,28cyclictest20198-21cpu09:48:520
17880997053,11cyclictest2952-21systemd-journal11:03:352
17880997053,11cyclictest2952-21systemd-journal11:03:352
17880997050,14cyclictest21025-21idleruntime-cro11:38:352
17880997050,14cyclictest21025-21idleruntime-cro11:38:352
17880997045,8cyclictest14173-21apt-get09:38:472
17880997031,10cyclictest181rcu_preempt10:17:142
17880997018,13cyclictest181rcu_preempt08:13:522
17880997018,13cyclictest181rcu_preempt08:13:522
17872997056,10cyclictest17858-21cyclictest08:46:021
17872997054,10cyclictest17858-21cyclictest10:08:571
17872997054,10cyclictest17858-21cyclictest10:08:571
1787299705,33cyclictest31-21ksoftirqd/110:30:181
1787299705,33cyclictest31-21ksoftirqd/110:30:181
1787299704,43cyclictest16952-21cpu08:38:511
1787299703,48cyclictest18312-21cut07:03:571
1787299703,26cyclictest31-21ksoftirqd/109:34:011
1787299702,27cyclictest31-21ksoftirqd/108:14:421
1787299702,27cyclictest31-21ksoftirqd/108:14:421
17865997053,11cyclictest7747-21sh10:22:230
17865997030,29cyclictest0-21swapper/010:55:090
1786599702,34cyclictest30136-21ntp_kernel_pll_10:59:030
1786599702,34cyclictest30136-21ntp_kernel_pll_10:59:030
17865997015,34cyclictest19697-21cpuspeed07:08:420
17865997011,20cyclictest7122-21cstates12:08:510
17865997011,20cyclictest7122-21cstates12:08:510
17880996940,7cyclictest181rcu_preempt09:24:302
1787299698,36cyclictest31-21ksoftirqd/107:13:281
1787299693,23cyclictest31-21ksoftirqd/107:45:201
1787299692,30cyclictest31-21ksoftirqd/107:54:081
1787299692,27cyclictest31-21ksoftirqd/112:00:391
17872996910,42cyclictest31-21ksoftirqd/111:25:471
17872996910,42cyclictest31-21ksoftirqd/111:25:471
17865996955,10cyclictest17858-21cyclictest08:29:100
17865996952,12cyclictest17858-21cyclictest11:44:300
17865996952,12cyclictest17858-21cyclictest11:44:300
17865996948,11cyclictest0-21swapper/008:18:350
17865996948,11cyclictest0-21swapper/008:18:350
17865996938,11cyclictest181rcu_preempt08:08:390
17865996925,32cyclictest22632-21timerwakeupswit08:54:110
17865996925,32cyclictest22632-21timerwakeupswit08:54:110
17880996840,8cyclictest26120-21ssh09:58:512
17880996834,10cyclictest0-21swapper/209:53:502
1788099683,36cyclictest4199-21fschecks_count07:58:562
17880996827,13cyclictest181rcu_preempt08:23:562
17880996824,20cyclictest7771-21ntp_states08:09:052
17880996824,20cyclictest7771-21ntp_states08:09:052
1787299684,50cyclictest5307-21ssh09:23:561
1787299684,34cyclictest31-21ksoftirqd/109:21:461
17872996836,23cyclictest30592-21grep10:59:111
17872996836,23cyclictest30592-21grep10:59:111
17872996832,27cyclictest16067-21grep08:34:101
1787299682,34cyclictest31-21ksoftirqd/108:53:391
1787299682,20cyclictest31-21ksoftirqd/112:30:001
1787299682,20cyclictest31-21ksoftirqd/112:30:001
17865996819,11cyclictest15764-21ssh10:34:470
17880996734,24cyclictest29271-21grep07:34:122
1787299674,36cyclictest7498-21fschecks_time08:08:581
1787299674,36cyclictest7498-21fschecks_time08:08:581
1787299672,28cyclictest31-21ksoftirqd/111:05:591
1787299672,28cyclictest31-21ksoftirqd/111:05:591
17865996736,11cyclictest181rcu_preempt07:33:470
1786599673,30cyclictest5256-21apt-get09:23:510
1786599673,24cyclictest3852-21latency_hist12:03:380
1788899669,41cyclictest0-21swapper/309:23:543
1788899667,38cyclictest0-21swapper/311:43:563
1788899667,38cyclictest0-21swapper/311:43:563
17880996616,43cyclictest381ktimers/207:28:492
17872996645,11cyclictest0-21swapper/107:58:561
1787299662,27cyclictest759-21acpi07:48:451
17872996621,22cyclictest31-21ksoftirqd/107:39:451
17865996643,9cyclictest181rcu_preempt07:32:450
1786599663,49cyclictest23070-21apt-get07:18:360
1786599663,32cyclictest10236-21apt-get12:13:470
1786599662,21cyclictest32339-21fschecks_time10:08:570
1786599662,21cyclictest32339-21fschecks_time10:08:570
17865996616,22cyclictest15-21ksoftirqd/007:48:390
17880996550,10cyclictest19780-21ntp_kernel_err12:29:032
17880996550,10cyclictest19780-21ntp_kernel_err12:29:032
1788099654,21cyclictest19037-21ntp_kernel_err08:44:032
17880996531,12cyclictest181rcu_preempt07:19:592
17880996531,12cyclictest181rcu_preempt07:19:592
17880996527,9cyclictest181rcu_preempt10:08:362
17865996545,11cyclictest0-21swapper/007:08:380
17865996531,8cyclictest181rcu_preempt10:23:520
17865996529,14cyclictest25673-21sed07:24:030
1786599652,32cyclictest26068-21/usr/sbin/munin09:58:570
1786599652,30cyclictest21515-21sendmail08:53:360
1788899649,32cyclictest0-21swapper/307:18:503
1788899649,32cyclictest0-21swapper/307:18:503
1788899643,41cyclictest4707-21chrt09:23:353
17880996451,9cyclictest58-21kcompactd007:13:572
17880996448,9cyclictest17858-21cyclictest07:39:112
17880996445,13cyclictest21966-21forks08:53:572
17880996445,13cyclictest21966-21forks08:53:572
1788099643,27cyclictest32068-21load07:44:012
17880996419,11cyclictest181rcu_preempt08:19:132
17880996330,7cyclictest181rcu_preempt08:38:372
17880996320,12cyclictest181rcu_preempt07:09:132
17865996326,10cyclictest0-21swapper/008:41:380
1786599632,29cyclictest25897-21latency_hist09:58:380
17888996213,33cyclictest0-21swapper/309:10:043
17888996213,33cyclictest0-21swapper/309:10:043
1788899614,40cyclictest23063-21chrt10:48:353
1788899613,42cyclictest17268-21taskset08:38:583
1788899613,40cyclictest2935-21taskset11:08:383
1788899613,40cyclictest2935-21taskset11:08:383
1788899612,39cyclictest12008-21taskset12:16:063
17888996111,41cyclictest0-21swapper/309:14:123
17880996127,8cyclictest181rcu_preempt08:38:462
17865996138,8cyclictest181rcu_preempt08:03:380
1788899603,44cyclictest19700-21taskset10:42:313
1788899603,44cyclictest19700-21taskset10:42:313
1788899603,42cyclictest8502-21chrt10:23:463
1788899594,39cyclictest2756-21taskset07:53:593
1788899594,33cyclictest11851-21chrt08:23:353
1788899582,43cyclictest30728-21taskset07:39:093
1788899582,40cyclictest27401-21chrt09:07:383
1708925813,30sleep30-21swapper/306:59:193
1788899574,35cyclictest32539-21taskset07:44:113
1788899574,35cyclictest31323-21chrt11:01:043
1788899574,35cyclictest31323-21chrt11:01:043
1788899574,35cyclictest13971-21taskset09:38:383
1788899573,38cyclictest22180-21chrt08:54:013
1788899573,38cyclictest22180-21chrt08:54:013
1788899572,38cyclictest20884-21chrt11:38:163
1788899572,38cyclictest20884-21chrt11:38:163
17888995718,27cyclictest7151-21chrt10:20:423
1788899564,37cyclictest5692-21chrt12:05:493
1788899564,34cyclictest28961-21chrt10:03:393
1788899563,36cyclictest8308-21taskset09:28:533
1788899562,41cyclictest25674-21taskset09:58:203
1788899554,39cyclictest22915-21taskset11:40:353
1788899554,34cyclictest10280-21chrt08:18:373
1788899554,34cyclictest10280-21chrt08:18:373
1788899553,38cyclictest1016-21chrt11:58:433
1788899553,36cyclictest26762-21taskset07:28:443
17888995518,23cyclictest22339-21taskset07:14:073
17888995518,22cyclictest16367-21chrt12:23:533
1788899547,31cyclictest15053-21chrt11:28:353
1788899547,31cyclictest15053-21chrt11:28:353
1788899544,37cyclictest15514-21chrt12:22:353
1788899543,35cyclictest14342-21chrt10:33:393
1788899543,35cyclictest14342-21chrt10:33:393
1788899543,34cyclictest12771-21taskset08:24:083
17888995416,25cyclictest29202-21taskset11:50:533
1788899533,37cyclictest16326-21chrt11:29:123
1788899533,35cyclictest15279-21chrt09:39:153
1788899533,32cyclictest20629-21chrt12:30:053
1788899533,32cyclictest20629-21chrt12:30:053
1788899532,38cyclictest23376-21taskset10:48:473
1788899532,38cyclictest23376-21taskset10:48:473
1788899532,37cyclictest18978-21chrt09:46:163
17888995318,23cyclictest10646-21chrt11:20:013
17888995318,21cyclictest4580-21chrt10:16:493
17888995318,20cyclictest1056-21chrt10:10:203
17888995318,20cyclictest1056-21chrt10:10:203
17888995317,23cyclictest23897-21chrt08:59:033
17888995316,24cyclictest29071-21chrt10:03:473
17888995315,24cyclictest924-21taskset07:48:533
1788899524,33cyclictest9671-21chrt12:13:353
1788899524,33cyclictest9671-21chrt12:13:353
1788899524,32cyclictest8783-21taskset08:13:393
1788899524,32cyclictest8783-21taskset08:13:393
1788899522,35cyclictest954-21taskset11:58:393
1788899522,35cyclictest954-21taskset11:58:393
1788899522,32cyclictest4174-21chrt07:58:533
17888995216,23cyclictest18724-21taskset08:43:573
1788899513,35cyclictest20885-21chrt07:09:133
1788899513,33cyclictest20525-21chrt09:49:003
1788899513,33cyclictest20525-21chrt09:49:003
1788899503,31cyclictest9152-21chrt11:18:393
17888995018,21cyclictest29449-21chrt07:34:483
17888995016,20cyclictest19416-21chrt07:08:363
17888994918,20cyclictest6778-21chrt08:06:143
17888994815,21cyclictest20210-21chrt08:48:513
17888994815,21cyclictest14771-21chrt08:30:123
17888994814,23cyclictest15314-21taskset08:33:403
17888994715,23cyclictest16590-21chrt10:37:083
17888994616,19cyclictest26462-21chrt07:28:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional