You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 11:15
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7.osadl.org (updated Tue Jul 21, 2026 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8628212610,71sleep08147-21/usr/sbin/munin18:52:120
1378721160,3sleep20-21swapper/220:46:352
1843321080,9sleep30-21swapper/323:42:003
898821039,22sleep30-21swapper/318:52:233
965599987,61cyclictest32542-21apt-get21:22:431
965599987,61cyclictest32542-21apt-get21:22:431
93342987,17sleep20-21swapper/218:54:382
9660999660,8cyclictest181rcu_preempt00:24:522
9655999455,13cyclictest181rcu_preempt21:36:481
9660999218,41cyclictest39-21ksoftirqd/220:16:392
9655999176,8cyclictest0-21swapper/120:27:001
9655999176,8cyclictest0-21swapper/120:27:001
9660999060,18cyclictest11812-21sh23:31:352
9660999060,18cyclictest11812-21sh23:31:352
9660999057,9cyclictest181rcu_preempt19:32:112
9660999020,41cyclictest39-21ksoftirqd/219:37:142
9660999013,46cyclictest39-21ksoftirqd/218:56:532
9655999053,29cyclictest6097-21kworker/u17:3-rpciod23:42:001
9660998970,13cyclictest17054-21diskmemload22:41:552
9660998927,34cyclictest39-21ksoftirqd/219:01:442
966099862,71cyclictest32542-21apt-get21:22:412
966099862,71cyclictest32542-21apt-get21:22:412
9651998666,14cyclictest15413-21fschecks_time23:36:590
9651998666,14cyclictest15413-21fschecks_time23:36:590
9651998516,13cyclictest17899-21/usr/sbin/munin22:47:190
9651998516,13cyclictest17899-21/usr/sbin/munin22:47:190
9660998430,32cyclictest39-21ksoftirqd/221:51:532
9655998452,10cyclictest181rcu_preempt20:02:081
9655998441,11cyclictest181rcu_preempt19:51:491
9655998441,11cyclictest181rcu_preempt19:51:491
9651998464,15cyclictest12169-21latency_hist00:26:380
9660998368,10cyclictest9009-21latency_hist23:26:392
9660998358,16cyclictest5640-21cron21:31:352
9655998362,14cyclictest58-21kcompactd023:48:081
9651998363,14cyclictest300-21rs:main0
9651998363,14cyclictest300-21rs:main0
9660998264,13cyclictest29547-21latency_hist19:56:392
9660998264,13cyclictest29547-21latency_hist19:56:392
9660998217,36cyclictest39-21ksoftirqd/220:22:022
9655998257,16cyclictest9964-21ntp_offset00:22:051
9660998125,33cyclictest39-21ksoftirqd/221:16:562
9655998147,14cyclictest8720-21apt-get20:31:591
9651998163,11cyclictest1-21systemd20:46:350
9660998039,11cyclictest14177-21sensors_temp19:07:092
9660998039,11cyclictest14177-21sensors_temp19:07:092
9655998051,11cyclictest181rcu_preempt22:06:491
9651998056,16cyclictest9052-21latency_hist23:26:390
9651998055,19cyclictest9506-21interrupts22:32:010
9660997953,20cyclictest8473-21apt-key20:31:392
9660997953,20cyclictest8473-21apt-key20:31:392
9655997948,8cyclictest181rcu_preempt19:11:511
9655997948,8cyclictest181rcu_preempt19:11:511
9655997943,11cyclictest181rcu_preempt22:41:501
9655997941,8cyclictest181rcu_preempt21:30:411
9655997932,25cyclictest15370-21fschecks_time23:36:591
9655997932,25cyclictest15370-21fschecks_time23:36:591
9651997917,37cyclictest0-21swapper/023:42:020
9651997913,27cyclictest24243-21/usr/sbin/munin22:02:030
9651997913,27cyclictest24243-21/usr/sbin/munin22:02:030
9660997858,14cyclictest17703-21sh23:41:352
9660997858,14cyclictest17703-21sh23:41:352
9660997856,9cyclictest181rcu_preempt19:27:062
9660997856,7cyclictest181rcu_preempt23:34:482
9660997836,7cyclictest181rcu_preempt21:04:182
9660997834,11cyclictest12266-21fschecks_count22:36:582
9660997824,35cyclictest39-21ksoftirqd/223:47:052
9660997814,27cyclictest39-21ksoftirqd/223:52:022
9655997856,15cyclictest9642-21cyclictest21:59:461
9655997847,9cyclictest181rcu_preempt20:12:031
965599783,40cyclictest5645-21wc20:22:031
9655997828,9cyclictest0-21swapper/123:11:511
9655997828,9cyclictest0-21swapper/123:11:511
9651997860,12cyclictest277-21dbus-daemon20:36:360
9660997758,14cyclictest17054-21diskmemload00:01:592
9660997756,15cyclictest21496-21sh21:57:112
9660997750,8cyclictest181rcu_preempt00:13:382
9660997738,10cyclictest20455-21rm23:45:592
9660997736,9cyclictest181rcu_preempt21:01:182
9655997740,13cyclictest11034-21ssh22:34:371
9655997739,19cyclictest31-21ksoftirqd/122:11:461
965599773,35cyclictest6079-21modprobe00:16:341
9655997730,12cyclictest8054-21ssh00:18:481
9655997730,12cyclictest8054-21ssh00:18:481
9651997762,9cyclictest27855-21kworker/u17:0+rpciod21:17:120
9660997656,15cyclictest27862-21latency_hist19:51:382
9660997645,10cyclictest181rcu_preempt21:41:562
9660997643,10cyclictest181rcu_preempt19:21:362
9660997638,8cyclictest181rcu_preempt23:07:042
9655997656,13cyclictest297-21systemd-logind20:41:361
9655997656,11cyclictest1513-21ssh00:07:141
9655997656,11cyclictest1513-21ssh00:07:141
9655997632,9cyclictest181rcu_preempt23:32:011
9655997630,16cyclictest181rcu_preempt22:37:131
952127612,31sleep10-21swapper/118:56:371
9660997557,12cyclictest27891-21sh23:57:362
9660997557,12cyclictest27891-21sh23:57:362
9660997555,14cyclictest300-21rs:main2
9660997555,14cyclictest300-21rs:main2
9660997553,12cyclictest0-21swapper/222:12:092
9660997552,8cyclictest181rcu_preempt23:12:322
9660997552,8cyclictest181rcu_preempt23:12:322
9660997542,9cyclictest0-21swapper/222:25:592
9660997542,23cyclictest10173-21apt-key20:36:382
9660997538,11cyclictest3537-21perl23:16:582
9660997527,8cyclictest0-21swapper/220:51:472
9660997511,34cyclictest39-21ksoftirqd/221:31:582
9660997511,34cyclictest39-21ksoftirqd/221:31:582
9655997551,17cyclictest352-21runrttasks23:27:041
9655997551,17cyclictest352-21runrttasks23:27:041
9655997542,9cyclictest16680-21ssh21:49:151
9655997542,9cyclictest16680-21ssh21:49:151
965599752,52cyclictest29970-21cut19:57:011
9651997558,11cyclictest32592-21systemctl20:06:360
9651997551,8cyclictest0-21swapper/023:02:090
9651997551,8cyclictest0-21swapper/023:02:090
965199752,39cyclictest15-21ksoftirqd/021:51:120
965199752,39cyclictest15-21ksoftirqd/021:51:120
9651997520,21cyclictest12350-21fschecks_time22:36:590
9660997458,11cyclictest2374-21systemd-journal23:03:592
9660997458,11cyclictest2374-21systemd-journal23:03:592
966099743,58cyclictest352-21runrttasks20:37:052
9660997433,9cyclictest26885-21cstates22:06:542
9655997446,19cyclictest26574-21fschecks_count21:11:581
9651997458,11cyclictest9642-21cyclictest22:55:150
9651997455,13cyclictest11903-21latency_hist23:31:380
9651997455,13cyclictest11903-21latency_hist23:31:380
9651997453,15cyclictest12452-21sh21:42:060
9651997420,14cyclictest24645-21awk23:52:060
9651997419,26cyclictest18962-21mailstats19:22:100
9651997416,19cyclictest18024-21ssh20:57:090
9660997358,8cyclictest181rcu_preempt21:37:052
9660997334,11cyclictest58-21kcompactd000:21:052
9660997334,11cyclictest58-21kcompactd000:21:052
9660997321,15cyclictest0-21swapper/220:16:512
9660997321,15cyclictest0-21swapper/220:16:512
9660997317,28cyclictest39-21ksoftirqd/200:09:172
9660997317,28cyclictest39-21ksoftirqd/200:09:172
9655997357,10cyclictest9642-21cyclictest19:06:391
965599734,37cyclictest31-21ksoftirqd/121:07:111
9655997336,16cyclictest181rcu_preempt22:18:401
9651997356,12cyclictest9642-21cyclictest21:02:510
9651997356,11cyclictest277-21dbus-daemon21:26:370
9651997356,11cyclictest277-21dbus-daemon21:26:370
9651997352,15cyclictest544-21cron19:41:360
965199734,51cyclictest14979-21latency_hist19:11:390
965199734,51cyclictest14979-21latency_hist19:11:390
9651997339,27cyclictest0-21swapper/022:12:050
965199732,41cyclictest0-21swapper/023:59:520
965199732,41cyclictest0-21swapper/023:59:520
9651997319,8cyclictest0-21swapper/019:01:500
9660997232,11cyclictest181rcu_preempt22:32:232
9660997217,21cyclictest39-21ksoftirqd/222:27:552
9660997216,25cyclictest39-21ksoftirqd/219:56:512
9655997256,10cyclictest8800-21awk22:31:351
9655997256,10cyclictest30676-21ssh00:02:101
9655997232,17cyclictest17805-21/usr/sbin/munin21:52:011
9655997230,18cyclictest181rcu_preempt20:18:421
9655997230,18cyclictest181rcu_preempt20:18:421
965599722,43cyclictest25358-21ssh22:03:421
965599722,43cyclictest25358-21ssh22:03:421
965199727,41cyclictest0-21swapper/023:01:010
965199727,41cyclictest0-21swapper/023:01:010
965199726,32cyclictest6203-21latency_hist00:16:370
965199726,32cyclictest10053-21awk21:37:160
965199726,23cyclictest12338-21/usr/sbin/munin23:31:580
9651997256,10cyclictest6954-21ssh22:28:000
9651997227,26cyclictest0-21swapper/000:04:170
9651997217,14cyclictest6054-21/usr/sbin/munin21:31:520
9651997217,14cyclictest6054-21/usr/sbin/munin21:31:520
9660997142,8cyclictest181rcu_preempt21:08:172
9655997141,12cyclictest181rcu_preempt21:06:011
9655997128,9cyclictest181rcu_preempt20:58:491
9651997152,14cyclictest20867-21cpuspeed21:56:530
9651997152,13cyclictest352-21runrttasks21:11:540
9651997133,14cyclictest0-21swapper/022:20:590
965199712,44cyclictest15-21ksoftirqd/023:11:290
9651997121,26cyclictest0-21swapper/023:13:180
9651997121,26cyclictest0-21swapper/023:13:180
9651997121,21cyclictest161ktimers/021:29:070
9651997118,14cyclictest14432-21grep20:47:010
9651997118,11cyclictest1196-21/usr/sbin/munin00:07:100
9651997118,11cyclictest1196-21/usr/sbin/munin00:07:100
9651997118,10cyclictest12358-21sed20:41:390
9660997056,8cyclictest15522-21open_inodes21:47:092
9660997056,8cyclictest15522-21open_inodes21:47:092
9660997050,15cyclictest31652-21kworker/u17:2-xprtiod22:19:382
9660997046,14cyclictest0-21swapper/222:46:552
9660997046,14cyclictest0-21swapper/222:46:552
966099703,28cyclictest15640-21ntp_states19:12:092
966099703,28cyclictest15640-21ntp_states19:12:092
9660997030,8cyclictest181rcu_preempt22:02:362
9660997030,8cyclictest181rcu_preempt22:02:362
9660997020,22cyclictest39-21ksoftirqd/220:47:102
9655997048,9cyclictest181rcu_preempt23:55:181
9655997034,13cyclictest181rcu_preempt20:42:341
9655997022,39cyclictest27434-21sh23:57:231
9655997022,39cyclictest27434-21sh23:57:231
9655997020,24cyclictest0-21swapper/119:31:391
965199706,31cyclictest26090-21cron19:46:350
965199706,31cyclictest26090-21cron19:46:350
9651997017,16cyclictest352-21runrttasks19:17:040
9651997017,13cyclictest23619-21ssh21:06:590
9651997015,17cyclictest2374-21systemd-journal22:06:410
9660996939,9cyclictest181rcu_preempt19:21:472
9660996925,34cyclictest381ktimers/222:52:052
9660996911,33cyclictest39-21ksoftirqd/220:01:482
9655996942,18cyclictest6768-21ntp_offset23:22:051
9655996936,11cyclictest181rcu_preempt21:41:511
9655996934,12cyclictest181rcu_preempt19:46:381
9655996934,12cyclictest181rcu_preempt19:46:381
9655996931,13cyclictest181rcu_preempt21:17:091
9655996928,16cyclictest14905-21timerwakeupswit20:47:111
9655996914,21cyclictest31-21ksoftirqd/123:17:111
965199695,24cyclictest19946-21apt-get19:26:530
965199694,25cyclictest0-21swapper/020:26:470
965199694,25cyclictest0-21swapper/020:26:470
965199693,55cyclictest21518-21latency23:47:010
965199693,23cyclictest16181-21ntpq20:52:050
965199692,28cyclictest15-21ksoftirqd/022:42:090
9651996914,16cyclictest31050-21runrttasks20:01:380
9651996911,15cyclictest16353-21chrt19:15:390
9651996911,15cyclictest16353-21chrt19:15:390
9660996821,19cyclictest277-21dbus-daemon19:46:362
9660996821,19cyclictest277-21dbus-daemon19:46:362
9655996836,14cyclictest3085-21ssh22:21:431
9655996833,8cyclictest181rcu_preempt19:37:131
9651996848,14cyclictest382-21/usr/sbin/munin20:07:050
965199684,31cyclictest18901-21unixbench_singl21:52:130
9651996817,13cyclictest5450-21/usr/sbin/munin20:21:590
965199681,19cyclictest15-21ksoftirqd/000:21:160
965199681,19cyclictest15-21ksoftirqd/000:21:160
9660996728,10cyclictest181rcu_preempt20:06:472
9655996753,9cyclictest0-21swapper/122:48:571
9655996753,9cyclictest0-21swapper/122:48:571
9655996735,8cyclictest181rcu_preempt23:01:531
9655996735,8cyclictest181rcu_preempt23:01:531
965599673,45cyclictest27734-21modprobe19:51:351
965599673,41cyclictest17466-21timerandwakeup19:17:121
9655996733,12cyclictest181rcu_preempt22:52:051
965599672,25cyclictest10042-21sort18:56:581
9655996714,19cyclictest0-21swapper/121:31:581
9655996714,19cyclictest0-21swapper/121:31:581
965199675,33cyclictest5768-21apt-get22:26:360
965199674,38cyclictest3186-21/usr/sbin/munin23:16:570
965199673,47cyclictest28742-21switchtime19:52:110
965199673,47cyclictest28742-21switchtime19:52:110
965199673,29cyclictest3401-21grep20:16:380
9651996714,15cyclictest10606-21timerandwakeup18:57:110
9660996641,11cyclictest181rcu_preempt21:13:142
9651996615,15cyclictest26760-21sed19:47:020
965599643,41cyclictest16241-21wc20:52:091
965599643,25cyclictest29951-21ssh23:06:571
9655996428,7cyclictest181rcu_preempt23:00:231
9655996428,7cyclictest181rcu_preempt23:00:231
9665996334,20cyclictest12548-21chrt23:32:023
965599633,41cyclictest19097-21timerandwakeup19:22:121
9665996249,9cyclictest451rcuc/322:01:213
965599623,34cyclictest14837-21idleruntime-cro19:11:361
965599623,34cyclictest14837-21idleruntime-cro19:11:361
9655996230,9cyclictest181rcu_preempt20:07:051
9655996226,12cyclictest181rcu_preempt19:36:271
965199625,26cyclictest15-21ksoftirqd/019:36:360
9665995739,13cyclictest451rcuc/319:08:303
9665995739,13cyclictest451rcuc/319:08:303
966599572,38cyclictest32601-21chrt23:11:393
9665995644,8cyclictest451rcuc/321:56:343
9665995642,9cyclictest451rcuc/323:15:533
9665995642,9cyclictest451rcuc/323:15:533
966599563,42cyclictest7188-21taskset00:17:103
966599563,42cyclictest7188-21taskset00:17:103
966599563,40cyclictest2432-21taskset00:09:403
966599563,40cyclictest2432-21taskset00:09:403
966599555,38cyclictest6487-21taskset22:27:353
9665995541,9cyclictest451rcuc/320:36:203
966599553,39cyclictest4304-21taskset23:17:143
966599553,39cyclictest25122-21taskset22:57:523
966599553,39cyclictest25122-21taskset22:57:523
966599552,40cyclictest10268-21chrt18:57:033
966599543,36cyclictest17680-21chrt21:51:393
966599543,36cyclictest17680-21chrt21:51:393
966599542,39cyclictest8745-21chrt21:36:393
966599542,39cyclictest8745-21chrt21:36:393
966599542,36cyclictest31520-21taskset20:02:003
966599533,36cyclictest16707-21chrt20:53:003
966599533,36cyclictest15132-21chrt22:41:553
966599533,35cyclictest6859-21taskset23:22:093
9665995238,9cyclictest451rcuc/320:08:583
9665995238,9cyclictest451rcuc/319:01:543
9665995235,10cyclictest451rcuc/321:27:013
966599523,38cyclictest32633-21taskset22:16:483
966599523,36cyclictest22262-21chrt19:32:113
966599523,35cyclictest17709-21taskset23:41:353
966599523,35cyclictest17709-21taskset23:41:353
966599523,34cyclictest6211-21chrt00:16:373
966599523,34cyclictest10181-21chrt23:27:123
966599523,34cyclictest10181-21chrt23:27:123
966599522,39cyclictest1120-21taskset21:23:133
966599522,39cyclictest1120-21taskset21:23:133
966599522,38cyclictest26559-21taskset21:11:573
966599522,37cyclictest6092-21chrt20:22:133
966599522,36cyclictest12086-21chrt22:36:523
966599522,35cyclictest15575-21chrt20:51:383
9665995136,10cyclictest451rcuc/319:14:463
9665995136,10cyclictest451rcuc/319:14:463
966599513,36cyclictest27911-21chrt22:07:283
966599513,36cyclictest27670-21chrt23:57:283
966599513,36cyclictest27670-21chrt23:57:283
966599513,35cyclictest29424-21chrt19:56:373
966599513,35cyclictest29424-21chrt19:56:373
966599513,35cyclictest18536-21taskset19:21:583
966599513,26cyclictest22785-21taskset21:05:573
966599504,31cyclictest23646-21taskset22:56:383
966599503,36cyclictest12112-21chrt20:41:383
966599503,34cyclictest17034-21chrt19:17:003
966599502,38cyclictest24324-21chrt22:02:053
966599502,38cyclictest24324-21chrt22:02:053
966599502,37cyclictest28604-21chrt23:04:213
966599502,37cyclictest28604-21chrt23:04:213
966599502,36cyclictest23130-21chrt23:50:023
966599502,35cyclictest23716-21chrt19:37:093
966599494,31cyclictest9813-21chrt22:32:103
9665994935,9cyclictest451rcuc/319:57:363
966599493,32cyclictest24868-21chrt21:08:213
966599493,32cyclictest19835-21chrt22:49:373
966599493,32cyclictest19835-21chrt22:49:373
966599492,35cyclictest18562-21taskset20:57:263
966599484,34cyclictest25717-21chrt23:54:003
9665994834,9cyclictest451rcuc/320:20:163
9665994834,9cyclictest451rcuc/320:20:163
9665994834,8cyclictest451rcuc/319:46:053
9665994834,8cyclictest451rcuc/319:46:053
966599483,34cyclictest29940-21chrt22:11:583
966599482,34cyclictest21374-21chrt19:31:383
9665994733,9cyclictest451rcuc/319:50:143
966599473,30cyclictest12474-21chrt20:42:153
966599472,34cyclictest14158-21chrt21:45:353
966599463,33cyclictest8912-21taskset21:36:473
9665994533,7cyclictest451rcuc/320:13:273
966599453,31cyclictest12001-21chrt00:26:343
9665994532,8cyclictest451rcuc/320:29:233
9665994532,8cyclictest451rcuc/320:29:233
966599444,30cyclictest4809-21chrt22:23:543
9665994429,10cyclictest451rcuc/300:02:243
9665994330,9cyclictest451rcuc/321:18:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional