You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 08:47
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7.osadl.org (updated Wed Apr 01, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2309021096,16sleep00-21swapper/019:03:050
233259910638,33cyclictest0-21swapper/122:59:341
233259910564,34cyclictest277-21dbus-daemon23:29:061
233399910257,38cyclictest14286-21gpgv23:19:082
2323621017,64sleep223305-21latency_hist19:04:092
23325991003,31cyclictest29550-21ssh22:49:351
23310991005,72cyclictest15-21ksoftirqd/020:49:310
23339999839,53cyclictest11002-21sh22:19:262
23339999839,53cyclictest11002-21sh22:19:262
2332599963,29cyclictest352-21runrttasks21:03:341
2332599962,30cyclictest352-21runrttasks22:55:311
229572956,15sleep10-21swapper/119:01:191
23339999469,18cyclictest373-21lldpd22:25:402
23339999446,41cyclictest0-21swapper/221:34:062
23325999451,36cyclictest544-21cron19:09:091
2332599943,42cyclictest20670-21ssh00:24:461
23339999356,31cyclictest26341-21fschecks_time22:44:282
2332599932,29cyclictest352-21runrttasks20:22:511
23310999321,39cyclictest15-21ksoftirqd/019:11:360
2332599922,27cyclictest1379550ssh23:17:491
2333999913,31cyclictest7869-21ntp_kernel_pll_21:19:342
2333999913,31cyclictest7869-21ntp_kernel_pll_21:19:342
23339999130,52cyclictest0-21swapper/220:34:432
2333999912,29cyclictest669350ssh22:11:562
2331099912,20cyclictest15-21ksoftirqd/022:24:210
23310999121,38cyclictest15-21ksoftirqd/022:03:290
2333999904,28cyclictest352-21runrttasks22:03:082
23339999036,47cyclictest0-21swapper/221:39:432
23339999036,47cyclictest0-21swapper/221:39:432
2333999903,32cyclictest16502-21fschecks_count21:34:262
2332599903,7cyclictest0-21swapper/121:19:181
2332599903,7cyclictest0-21swapper/121:19:181
2333999894,41cyclictest19676-21snmpd20:59:022
2333999893,45cyclictest24973-21head20:44:412
2333999893,33cyclictest8394-21fschecks_count00:04:222
2333999893,33cyclictest8394-21fschecks_count00:04:222
23339998923,57cyclictest30622-21latency_hist21:04:092
23325998944,38cyclictest14037-21latency_hist20:14:091
23325998944,38cyclictest14037-21latency_hist20:14:091
23339998872,10cyclictest3969-21sendmail21:14:052
2333999883,31cyclictest25815-21fschecks_time19:09:302
23339998818,24cyclictest2847-21modprobe23:54:062
23339998818,24cyclictest2847-21modprobe23:54:062
23325998843,35cyclictest0-21swapper/121:04:281
23310998863,16cyclictest30452-21diskmemload23:04:410
23339998765,15cyclictest28500-21fschecks_count21:54:272
2333999873,33cyclictest7939-21sort00:04:072
2333999873,30cyclictest30452-21diskmemload21:48:172
23339998725,56cyclictest0-21swapper/219:09:082
23339998725,56cyclictest0-21swapper/219:09:082
23325998742,35cyclictest0-21swapper/122:24:151
23339998633,46cyclictest0-21swapper/220:04:232
2333999863,33cyclictest7338-21modprobe19:54:052
2333999863,32cyclictest28877-21seq19:19:232
2333999863,31cyclictest9416-21fschecks_count19:59:282
2333999863,31cyclictest27956-21idleruntime-cro21:54:062
2333999863,30cyclictest20725-21cp23:29:082
2333999863,30cyclictest18862-21latency_hist20:29:092
2333999862,33cyclictest17464-21sh00:19:282
2333999862,33cyclictest13355-21tail20:09:422
2333999862,33cyclictest13355-21tail20:09:422
2333999862,28cyclictest9058-21chrt19:59:082
23325998640,39cyclictest8293-21sh23:09:081
23325998640,39cyclictest14788-21/usr/sbin/munin23:20:441
23325998640,39cyclictest14788-21/usr/sbin/munin23:20:441
23325998630,46cyclictest0-21swapper/121:44:211
23310998636,24cyclictest181rcu_preempt22:43:480
23310998636,24cyclictest181rcu_preempt22:43:480
2333999854,34cyclictest22703-21modprobe22:39:052
2333999853,34cyclictest5424-21latency_hist23:04:082
2333999853,34cyclictest24491-21cut23:34:422
2333999853,32cyclictest21491-21switchtime00:25:102
2333999853,32cyclictest1527-21cut19:34:312
2333999853,32cyclictest1527-21cut19:34:312
2333999853,31cyclictest4288-21/usr/sbin/munin21:14:432
23339998532,46cyclictest0-21swapper/223:54:382
23339998532,46cyclictest0-21swapper/223:54:382
2333999852,32cyclictest17047-21ssh22:29:222
2333999852,32cyclictest17047-21ssh22:29:222
2333999852,19cyclictest295250systemd-journal22:06:302
2333999852,19cyclictest295250systemd-journal22:06:302
23325998566,13cyclictest2666-21threads22:04:411
23325998566,13cyclictest2666-21threads22:04:411
23325998542,37cyclictest16984-21sh21:34:381
23325998542,34cyclictest19131-21fschecks_time20:29:271
2332599853,31cyclictest20979-21cat20:34:471
23310998557,23cyclictest29433-21latency_hist23:44:080
2333999844,31cyclictest21517-21if_eth023:30:412
2333999844,30cyclictest26648-21/usr/sbin/munin23:39:392
2333999844,26cyclictest3680-21grep19:39:412
2333999844,26cyclictest3680-21grep19:39:412
2333999843,31cyclictest10723-21latency_hist22:19:092
2333999843,30cyclictest27796-21wc19:14:402
2333999843,27cyclictest4770-21wc19:44:312
2333999843,27cyclictest4770-21wc19:44:312
2333999843,27cyclictest11880-21grep00:09:392
2333999842,32cyclictest14793-21grep20:14:402
2333999842,30cyclictest23275-21grep22:39:272
2333999842,30cyclictest23275-21grep22:39:272
23325998462,17cyclictest23302-21cyclictest22:09:401
23325998441,36cyclictest17962-21mailstats20:24:391
23325998441,36cyclictest0-21swapper/122:44:281
2332599843,32cyclictest11375-21/usr/sbin/munin20:04:341
23325998430,30cyclictest22162-21latency_hist21:44:101
23325998430,30cyclictest22162-21latency_hist21:44:101
2332599842,30cyclictest5002-21seq23:56:471
2332599842,30cyclictest5002-21seq23:56:471
23310998461,16cyclictest19767-21ssh21:39:330
23310998461,16cyclictest19767-21ssh21:39:330
23310998458,18cyclictest11806-21rm23:14:280
2331099843,58cyclictest15-21ksoftirqd/021:03:220
2331099843,51cyclictest15-21ksoftirqd/019:06:420
2331099843,51cyclictest15-21ksoftirqd/019:06:420
23310998428,33cyclictest15-21ksoftirqd/019:16:090
23310998414,19cyclictest161ktimers/023:54:080
23310998414,19cyclictest161ktimers/023:54:080
2333999834,31cyclictest29166-21ssh22:49:252
2333999833,32cyclictest23253-21ls20:39:412
2333999833,31cyclictest11203-21cat21:24:422
2333999833,30cyclictest8873-21fschecks_time23:09:272
2333999833,29cyclictest31038-21ssh21:04:292
2333999833,28cyclictest7814-21runrttasks23:07:542
2333999832,32cyclictest30378-21acpi19:24:172
2333999832,31cyclictest32444-21uname22:54:322
2333999832,23cyclictest15947-21runrttasks23:20:542
2333999832,23cyclictest15947-21runrttasks23:20:542
23325998342,34cyclictest0-21swapper/123:44:251
23325998341,36cyclictest0-21swapper/119:34:081
2332599833,7cyclictest0-21swapper/122:39:171
2332599833,7cyclictest0-21swapper/122:39:171
2332599833,32cyclictest23839-21/usr/sbin/munin19:04:311
2332599833,32cyclictest23839-21/usr/sbin/munin19:04:311
2331099833,41cyclictest15-21ksoftirqd/021:31:110
2331099832,56cyclictest15-21ksoftirqd/020:34:140
2331099832,50cyclictest15-21ksoftirqd/020:18:420
23310998324,42cyclictest15-21ksoftirqd/019:41:120
23310998324,42cyclictest15-21ksoftirqd/019:41:120
2333999823,33cyclictest16363-21sshd00:17:322
2333999823,31cyclictest32639-21cut19:29:392
2333999823,30cyclictest18894-21/usr/sbin/munin20:29:392
2333999823,28cyclictest32436-21latency_hist23:49:082
23325998242,34cyclictest30452-21diskmemload00:01:551
23325998241,35cyclictest30452-21diskmemload21:18:001
23325998239,36cyclictest0-21swapper/120:50:251
23325998238,38cyclictest29290-21ntp_kernel_pll_19:19:351
2332599823,33cyclictest2858-40ogrotate23:54:061
2332599823,33cyclictest2858-40ogrotate23:54:061
2332599823,31cyclictest26850-21sh23:39:231
2332599823,31cyclictest22403-21/usr/sbin/munin20:39:221
2332599822,31cyclictest12629-21seq22:21:411
2332599822,31cyclictest12629-21seq22:21:411
23310998241,31cyclictest0-21swapper/023:44:260
23310998226,31cyclictest15-21ksoftirqd/022:13:100
23310998218,39cyclictest15-21ksoftirqd/019:45:580
23310998218,39cyclictest15-21ksoftirqd/019:45:580
23310998215,44cyclictest15-21ksoftirqd/020:22:230
2334499813,37cyclictest8588-21chrt23:09:203
2332599813,7cyclictest0-21swapper/100:29:181
23325998137,38cyclictest31229-21timerwakeupswit19:24:421
23325998134,39cyclictest0-21swapper/121:29:091
2332599813,36cyclictest32068-21/usr/sbin/munin21:59:411
2332599813,32cyclictest24001-21/usr/sbin/munin20:44:161
2332599813,32cyclictest14898-21/usr/sbin/munin00:14:371
2332599812,41cyclictest28256-21apt-get21:54:171
2332599812,34cyclictest17682-21/usr/sbin/munin00:19:321
2331099815,46cyclictest15-21ksoftirqd/021:06:590
23310998153,22cyclictest23734-21fschecks_count00:29:270
2333999803,30cyclictest15877-21fschecks_count20:19:292
23339998029,43cyclictest0-21swapper/200:29:192
2333999802,31cyclictest26138-21fschecks_time20:50:182
2332599803,6cyclictest0-21swapper/119:59:231
2332599803,32cyclictest8097-21/usr/sbin/munin19:54:321
2332599803,32cyclictest13664-21/usr/sbin/munin21:29:301
2332599803,31cyclictest6557-21/usr/sbin/munin19:49:401
2332599803,29cyclictest15590-21latency_hist20:19:081
2332599802,36cyclictest25294-21cpu21:49:231
23310998012,46cyclictest15-21ksoftirqd/000:19:270
23325997939,35cyclictest0-21swapper/100:04:221
23325997939,35cyclictest0-21swapper/100:04:221
2332599793,36cyclictest1394-21cpuspeed21:09:231
2332599793,31cyclictest20257-21ssh22:34:261
2332599792,6cyclictest0-21swapper/119:14:191
2332599792,32cyclictest11536-21grep00:09:291
2332599792,31cyclictest19195-21seq22:32:441
2332599792,31cyclictest19195-21seq22:32:441
2332599792,31cyclictest1677-21/usr/sbin/munin19:34:341
2332599792,31cyclictest1677-21/usr/sbin/munin19:34:341
23310997956,15cyclictest2952-21systemd-journal23:10:380
2331099795,46cyclictest15-21ksoftirqd/022:16:220
2331099793,47cyclictest15-21ksoftirqd/020:40:170
2331099792,51cyclictest15-21ksoftirqd/022:55:000
2331099792,46cyclictest15-21ksoftirqd/020:44:230
2331099792,45cyclictest15-21ksoftirqd/021:49:080
2331099792,37cyclictest15-21ksoftirqd/023:24:050
2331099792,37cyclictest15-21ksoftirqd/023:24:050
2332599784,34cyclictest2904-21expr19:39:221
2332599784,34cyclictest2904-21expr19:39:221
2332599783,32cyclictest23892-21sh23:34:261
2332599782,33cyclictest8476-21/usr/sbin/munin22:14:381
2332599782,31cyclictest9067-21/usr/sbin/munin23:09:311
23310997853,16cyclictest23987-21ssh23:34:300
2331099783,53cyclictest20482-21/usr/sbin/munin00:24:160
2331099783,43cyclictest15-21ksoftirqd/000:01:220
2331099781,46cyclictest15-21ksoftirqd/020:26:120
23310997810,20cyclictest15-21ksoftirqd/019:29:110
2332599773,30cyclictest27646-21/usr/sbin/munin20:54:291
2332599772,31cyclictest21054-21taskset23:29:241
2332599772,28cyclictest4395-21cpu19:44:221
2332599772,28cyclictest4395-21cpu19:44:221
23310997758,12cyclictest29504-21runrttasks19:19:410
2331099774,35cyclictest15-21ksoftirqd/022:22:580
2331099774,35cyclictest15-21ksoftirqd/022:22:580
2331099773,42cyclictest15-21ksoftirqd/021:12:500
23310997733,34cyclictest0-21swapper/021:14:300
2331099772,44cyclictest15-21ksoftirqd/019:37:030
2331099772,44cyclictest15-21ksoftirqd/019:37:030
23310997724,37cyclictest15-21ksoftirqd/020:30:080
2331099765,36cyclictest1-21systemd23:34:060
2331099764,51cyclictest15-21ksoftirqd/023:55:240
2331099764,51cyclictest15-21ksoftirqd/023:55:240
2331099764,46cyclictest15-21ksoftirqd/021:56:430
2331099763,35cyclictest10868-21ntp_kernel_pll_21:24:350
23310997621,23cyclictest15-21ksoftirqd/019:55:340
23310997610,47cyclictest15-21ksoftirqd/022:04:330
23310997610,47cyclictest15-21ksoftirqd/022:04:330
23310997553,16cyclictest14558-21fschecks_time00:14:270
23310997549,17cyclictest7473-21sh21:19:240
23310997549,17cyclictest7473-21sh21:19:240
23310997548,18cyclictest30452-21diskmemload23:26:490
2331099753,35cyclictest15-21ksoftirqd/021:51:380
2331099753,33cyclictest5247-21cron23:04:050
2331099752,46cyclictest15-21ksoftirqd/020:14:050
2331099752,46cyclictest15-21ksoftirqd/020:14:050
23310997518,32cyclictest15-21ksoftirqd/022:50:500
2334499743,34cyclictest27327-21chrt22:45:163
23310997435,17cyclictest181rcu_preempt00:05:170
23310997435,17cyclictest181rcu_preempt00:05:170
2331099743,49cyclictest15-21ksoftirqd/022:33:280
2331099743,49cyclictest15-21ksoftirqd/022:33:280
2331099743,42cyclictest19108-21latency_hist21:39:090
2331099743,39cyclictest15-21ksoftirqd/020:08:580
2334499733,26cyclictest32415-21chrt23:49:083
2334499733,25cyclictest22851-21chrt21:44:353
2334499732,30cyclictest24614-21taskset20:44:373
2331099733,35cyclictest28694-21latency_hist22:49:050
2331099722,38cyclictest15-21ksoftirqd/020:00:280
23310997220,27cyclictest15-21ksoftirqd/019:29:050
2334499713,26cyclictest11217-21chrt00:09:193
2334499712,24cyclictest9221-21chrt19:59:173
23310997127,19cyclictest15-21ksoftirqd/000:13:500
2334499704,25cyclictest29105-21taskset20:59:203
2334499703,31cyclictest30301-21chrt22:50:153
2334499703,29cyclictest27181-21chrt23:39:313
2334499703,27cyclictest31362-21chrt19:24:433
2334499703,26cyclictest18154-21chrt20:24:413
2334499703,25cyclictest1737-21taskset22:04:093
2334499703,25cyclictest1737-21taskset22:04:093
2334499703,25cyclictest16150-21chrt21:34:093
2334499694,24cyclictest27675-21chrt20:54:303
2334499693,30cyclictest10467-21chrt21:24:243
2334499693,26cyclictest7539-21chrt22:14:063
2334499693,26cyclictest15947-21chrt23:20:543
2334499693,26cyclictest15947-21chrt23:20:543
2334499693,25cyclictest2228-21chrt19:35:363
2334499693,25cyclictest2228-21chrt19:35:363
2334499693,25cyclictest1805-21chrt21:09:333
2334499693,25cyclictest16306-21chrt22:28:013
2334499693,25cyclictest13661-21taskset22:24:083
2334499693,25cyclictest13661-21taskset22:24:083
2334499692,32cyclictest27289-21chrt21:52:223
23310996946,10cyclictest181rcu_preempt22:37:340
2331099694,43cyclictest7403-21latency_hist19:54:080
2334499684,29cyclictest9891-21taskset21:23:563
2334499684,29cyclictest9891-21taskset21:23:563
2334499683,27cyclictest5766-21taskset23:04:263
2334499682,29cyclictest4523-21taskset21:14:273
2334499674,23cyclictest469-21chrt23:49:283
2334499674,23cyclictest469-21chrt23:49:283
2334499673,25cyclictest19466-21chrt00:22:493
2334499673,23cyclictest28779-21chrt21:54:323
2331099672,46cyclictest15-21ksoftirqd/020:57:430
2334499663,26cyclictest18866-21taskset23:24:433
2334499663,26cyclictest18630-21chrt22:31:093
2334499663,26cyclictest18630-21chrt22:31:093
2334499644,25cyclictest10459-21taskset00:07:573
2334499644,25cyclictest10459-21taskset00:07:573
2334499643,42cyclictest917-21kworker/3:1+events21:34:143
2334499643,25cyclictest8229-21chrt22:14:313
2334499643,25cyclictest1823-21taskset22:57:533
2334499643,24cyclictest27460-21taskset19:14:303
2334499633,26cyclictest14926-21taskset00:14:383
2334499632,27cyclictest23068-21chrt23:33:193
2334499623,42cyclictest27183-21chrt20:54:053
2334499622,27cyclictest5213-21taskset23:04:013
2334499622,27cyclictest21397-21taskset22:35:273
2334499622,26cyclictest20409-21chrt20:34:083
2334499622,25cyclictest28789-21taskset19:19:203
23344996212,16cyclictest0-21swapper/322:04:083
23344996011,14cyclictest0-21swapper/322:39:343
23344996011,14cyclictest0-21swapper/322:39:343
109922600,6sleep30-21swapper/320:04:243
2334499593,35cyclictest0-21swapper/300:24:253
2334499592,26cyclictest436-21taskset21:07:383
2334499583,40cyclictest26506-21chrt19:09:443
23344995812,12cyclictest0-21swapper/323:56:433
23344995812,12cyclictest0-21swapper/323:56:433
2334499564,38cyclictest929-21chrt19:34:063
2334499563,37cyclictest25461-21taskset00:32:093
2334499554,35cyclictest4618-21taskset19:44:273
2334499554,35cyclictest4618-21taskset19:44:273
2334499542,36cyclictest21160-21chrt21:41:563
2334499542,36cyclictest21160-21chrt21:41:563
23344995314,30cyclictest0-21swapper/320:34:293
2334499523,35cyclictest23905-21taskset20:44:093
2334499514,32cyclictest13371-21chrt23:16:383
2334499504,32cyclictest8011-21chrt00:04:083
2334499493,34cyclictest6398-21chrt19:49:333
2210724910,24sleep30-21swapper/318:59:343
2334499484,31cyclictest3287-21taskset19:39:333
2334499484,31cyclictest3287-21taskset19:39:333
2334499484,31cyclictest13909-21chrt20:14:083
2334499484,31cyclictest13909-21chrt20:14:083
2334499483,32cyclictest23785-21taskset19:04:303
2334499483,32cyclictest23785-21taskset19:04:303
2334499482,31cyclictest16505-21chrt20:19:433
2334499463,29cyclictest23919-21taskset23:34:283
2334499453,30cyclictest15226-21taskset20:16:353
2334499453,28cyclictest7631-21taskset19:54:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional