You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 02:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Thu Apr 23, 2026 12:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23403217115,117sleep023458-21fschecks_count06:58:010
23791212312,81sleep223825-21proc_pri06:58:162
247289910957,47cyclictest24697-21cyclictest08:17:522
247289910957,47cyclictest24697-21cyclictest08:17:522
247289910243,12cyclictest0-21swapper/208:42:382
24728991023,43cyclictest30235-21latency_hist08:57:392
24708991014,75cyclictest21843-21apt-get10:32:510
24708991014,75cyclictest21843-21apt-get10:32:510
247289910037,12cyclictest0-21swapper/207:27:522
2472899976,31cyclictest21843-21apt-get10:33:402
2472899976,31cyclictest21843-21apt-get10:33:402
24728999735,8cyclictest181rcu_preempt11:37:532
2472899972,52cyclictest30193-21ssh09:53:152
24728999637,10cyclictest0-21swapper/211:32:512
24708999661,8cyclictest181rcu_preempt07:57:570
24708999661,8cyclictest181rcu_preempt07:57:570
232152958,57sleep123226-21acpi06:57:451
24708999253,33cyclictest24841-21/usr/sbin/munin10:38:000
2339929216,46sleep30-21swapper/306:58:003
24720999166,17cyclictest31372-21sh09:53:541
24708999164,18cyclictest30309-21latency_hist07:17:390
24720998866,16cyclictest4517-21sh11:52:101
24720998764,17cyclictest15582-21apt-key08:12:381
24720998764,17cyclictest15582-21apt-key08:12:381
2470899874,53cyclictest17755-21awk12:13:070
24708998661,17cyclictest1-21systemd10:24:110
24708998549,12cyclictest181rcu_preempt08:54:010
2472899843,52cyclictest12582-21ssh09:23:112
24720998441,32cyclictest19130-21modprobe11:22:341
24728998361,14cyclictest2374-21systemd-journal10:04:112
24728998355,19cyclictest17536-21unixbench_singl11:18:112
24708998343,12cyclictest0-21swapper/011:18:000
24728998242,19cyclictest25016-21kworker/u17:2-writeback08:42:562
24708998158,15cyclictest297-21systemd-logind08:12:360
24708998158,15cyclictest297-21systemd-logind08:12:360
24728998056,18cyclictest30975-21cstates10:47:522
24728998054,17cyclictest16400-21ssh10:23:092
2470899807,52cyclictest15-21ksoftirqd/010:27:480
24708998046,13cyclictest181rcu_preempt12:05:350
24728997957,15cyclictest5403-21latency_hist09:12:372
24720997960,13cyclictest3700-21unixbench_multi07:33:121
24708997933,7cyclictest181rcu_preempt11:58:180
24708997933,17cyclictest181rcu_preempt11:26:110
24708997855,16cyclictest2698-21latency_hist07:32:380
24708997818,29cyclictest161ktimers/011:52:500
24708997818,29cyclictest161ktimers/011:52:500
2472899774,39cyclictest7404-21/usr/sbin/munin11:02:582
24720997755,16cyclictest24697-21cyclictest11:59:221
24720997747,21cyclictest1535-21cstates10:52:511
24720997747,21cyclictest1535-21cstates10:52:511
24708997759,12cyclictest2374-21systemd-journal09:57:360
24708997757,14cyclictest23020-21ntp_kernel_pll_11:28:030
24708997757,14cyclictest23020-21ntp_kernel_pll_11:28:030
24708997746,16cyclictest181rcu_preempt07:06:000
24708997738,9cyclictest0-21swapper/007:17:580
24708997725,11cyclictest0-21swapper/008:32:550
2470899772,38cyclictest352-21runrttasks09:52:070
24720997651,18cyclictest31755-21idleruntime-cro07:22:361
24708997650,7cyclictest181rcu_preempt07:37:490
2472899753,49cyclictest29442-21latency_hist09:52:382
24720997561,10cyclictest24697-21cyclictest09:30:061
24720997561,10cyclictest24697-21cyclictest09:30:061
2470899753,55cyclictest14765-21mailstats12:08:100
24708997524,26cyclictest0-21swapper/010:43:060
24708997524,26cyclictest0-21swapper/010:43:060
24720997453,15cyclictest544-21cron11:12:351
24720997450,18cyclictest29593-21apt-key09:52:381
2472099743,33cyclictest31-21ksoftirqd/110:03:191
24720997423,37cyclictest22868-21sed11:28:001
24720997423,37cyclictest22868-21sed11:28:001
2470899746,37cyclictest15-21ksoftirqd/012:27:470
24708997449,20cyclictest31469-21kworker/u17:1+xprtiod12:22:380
24708997449,20cyclictest31469-21kworker/u17:1+xprtiod12:22:380
24708997439,8cyclictest181rcu_preempt10:03:000
24708997425,33cyclictest0-21swapper/010:07:480
2472899736,30cyclictest26567-21kworker/2:1+mm_percpu_wq09:03:052
24728997352,12cyclictest31469-21kworker/u17:1-rpciod11:50:432
24728997351,8cyclictest181rcu_preempt12:19:122
24728997351,8cyclictest181rcu_preempt12:19:122
24728997346,20cyclictest23759-21sh08:37:352
24728997344,10cyclictest26371-21latency_hist07:07:382
24720997349,15cyclictest21263-21diskmemload09:38:021
2472099732,35cyclictest31-21ksoftirqd/109:21:471
2472099732,35cyclictest31-21ksoftirqd/109:21:471
24708997353,13cyclictest15297-21ssh10:22:350
24708997343,12cyclictest181rcu_preempt09:45:200
24708997343,12cyclictest181rcu_preempt09:45:200
24708997341,11cyclictest0-21swapper/011:32:540
24728997250,15cyclictest2374-21systemd-journal11:23:562
24728997244,14cyclictest0-21swapper/209:13:112
24728997243,20cyclictest12901-21fschecks_time10:17:582
2472099728,55cyclictest27164-21apt-get07:08:051
24720997256,9cyclictest0-21swapper/111:02:341
24720997253,13cyclictest352-21runrttasks10:37:391
24720997248,8cyclictest181rcu_preempt09:43:241
24720997248,8cyclictest181rcu_preempt09:43:241
2472099724,39cyclictest31-21ksoftirqd/112:31:511
24720997242,21cyclictest32141-21processes11:43:071
24720997242,21cyclictest32141-21processes11:43:071
24720997228,9cyclictest0-21swapper/108:22:581
24720997228,9cyclictest0-21swapper/108:22:581
2470899725,43cyclictest15-21ksoftirqd/008:27:500
24708997236,16cyclictest181rcu_preempt11:50:540
24708997232,12cyclictest181rcu_preempt12:23:050
24728997150,8cyclictest181rcu_preempt12:00:172
24728997145,19cyclictest27583-21latency_hist10:42:372
2472899713,52cyclictest14523-21kworker/u17:2+ext4-rsv-conversion07:09:102
2472899713,42cyclictest28180-21idleruntime10:42:572
2472899713,42cyclictest28180-21idleruntime10:42:572
2472899712,40cyclictest17951-21cat12:13:092
24720997153,12cyclictest30428-21latency_hist08:57:391
24720997148,8cyclictest181rcu_preempt10:28:431
24720997143,20cyclictest0-21swapper/111:37:521
2472099713,34cyclictest8281-21cat07:48:091
2472099712,25cyclictest31-21ksoftirqd/111:33:411
24720997119,29cyclictest31-21ksoftirqd/107:53:001
24708997146,9cyclictest1423-21kworker/u17:0+xprtiod09:06:410
24708997144,9cyclictest30369-21ssh11:40:240
24708997143,17cyclictest181rcu_preempt08:38:440
24708997131,28cyclictest0-21swapper/010:48:220
2473499706,45cyclictest0-21swapper/309:53:153
24728997052,13cyclictest11561-21latency_hist09:22:382
24728997052,13cyclictest11561-21latency_hist09:22:382
24728997049,14cyclictest297-21systemd-logind10:17:352
24728997049,14cyclictest297-21systemd-logind10:17:352
24728997046,12cyclictest0-21swapper/210:53:112
24728997046,12cyclictest0-21swapper/210:53:112
24728997036,14cyclictest16702-21cron12:12:342
2472899702,32cyclictest347-21sshd07:23:122
2472899702,32cyclictest347-21sshd07:23:122
24720997054,11cyclictest2374-21systemd-journal10:11:111
24720997054,10cyclictest25734-21ssh12:27:341
24720997049,14cyclictest365-21ntpd09:13:051
24720997046,17cyclictest544-21cron08:47:351
2472099704,40cyclictest1798-21mailstats07:28:081
2472099702,35cyclictest31-21ksoftirqd/112:07:511
24708997054,10cyclictest6955-21ssh11:02:110
24708997047,9cyclictest181rcu_preempt09:28:000
24708997047,9cyclictest181rcu_preempt09:28:000
24708997047,17cyclictest2966-21fschecks_count09:07:580
24708997046,8cyclictest181rcu_preempt07:09:180
24708997041,11cyclictest181rcu_preempt11:10:410
24708997039,9cyclictest181rcu_preempt08:50:280
24708997038,16cyclictest181rcu_preempt08:23:100
24708997038,16cyclictest181rcu_preempt08:23:100
24708997032,11cyclictest181rcu_preempt08:13:500
24708997032,11cyclictest181rcu_preempt08:13:500
2470899702,35cyclictest6082-21/usr/sbin/munin09:12:590
24708997014,8cyclictest181rcu_preempt09:22:580
24728996954,11cyclictest29858-21kworker/u17:1-writeback07:38:102
24728996947,16cyclictest24697-21cyclictest09:33:012
24728996943,19cyclictest544-21cron08:07:352
24728996943,19cyclictest544-21cron08:07:352
24728996941,8cyclictest0-21swapper/210:09:132
24728996940,9cyclictest24697-21cyclictest09:45:022
24728996940,9cyclictest24697-21cyclictest09:45:022
24728996940,20cyclictest28760-21latency_hist08:52:382
24728996936,9cyclictest181rcu_preempt11:01:332
2472899693,38cyclictest23324-21fschecks_time12:22:572
24720996953,12cyclictest24697-21cyclictest12:03:001
24720996951,13cyclictest20320-21fschecks_time12:17:571
24720996951,13cyclictest20320-21fschecks_time12:17:571
24720996945,15cyclictest18470-21diskmemload09:33:081
24720996943,17cyclictest0-21swapper/107:58:001
24720996943,17cyclictest0-21swapper/107:58:001
2470899696,46cyclictest15-21ksoftirqd/009:58:110
2470899696,46cyclictest15-21ksoftirqd/009:58:110
24708996939,9cyclictest181rcu_preempt10:16:430
24708996939,9cyclictest181rcu_preempt10:16:430
24708996933,13cyclictest181rcu_preempt11:43:500
24708996933,13cyclictest181rcu_preempt11:43:500
24708996930,12cyclictest0-21swapper/008:58:330
2470899692,33cyclictest21309-21netstat09:38:030
2472899688,13cyclictest0-21swapper/211:42:492
2472899688,13cyclictest0-21swapper/211:42:492
24728996850,12cyclictest277-21dbus-daemon08:12:372
24728996850,12cyclictest277-21dbus-daemon08:12:372
24728996849,12cyclictest24697-21cyclictest07:12:572
24728996845,13cyclictest0-21swapper/212:32:372
24728996829,28cyclictest11447-21kernelversion12:02:592
24720996851,10cyclictest18306-21systemd-run10:27:361
24720996837,10cyclictest181rcu_preempt10:37:581
2472099683,45cyclictest30837-21/usr/sbin/munin10:48:071
2472099682,34cyclictest31-21ksoftirqd/112:12:591
2472099682,32cyclictest31-21ksoftirqd/110:45:241
2472099682,32cyclictest31-21ksoftirqd/110:45:241
2470899685,30cyclictest11433-21munin-plugin-st09:22:350
2470899685,30cyclictest11433-21munin-plugin-st09:22:350
2470899684,44cyclictest13863-21latency_hist08:07:370
2470899684,44cyclictest13863-21latency_hist08:07:370
24708996842,9cyclictest181rcu_preempt07:42:510
24708996816,43cyclictest161ktimers/009:34:000
24728996748,12cyclictest0-21swapper/209:40:172
24728996745,14cyclictest2374-21systemd-journal11:54:012
24728996745,14cyclictest2374-21systemd-journal11:54:012
24728996742,14cyclictest0-21swapper/207:22:382
24728996729,12cyclictest0-21swapper/210:01:092
24728996729,12cyclictest0-21swapper/210:01:092
24728996726,18cyclictest24697-21cyclictest07:33:072
24720996745,8cyclictest181rcu_preempt10:33:131
2472099674,35cyclictest4418-21cpu07:37:491
2472099672,46cyclictest352-21runrttasks07:03:211
2472099672,29cyclictest31-21ksoftirqd/108:17:161
2472099672,29cyclictest31-21ksoftirqd/108:17:161
2470899673,41cyclictest32749-21switchtime07:23:110
2470899673,41cyclictest32749-21switchtime07:23:110
24708996713,31cyclictest15-21ksoftirqd/007:49:340
24728996639,13cyclictest181rcu_preempt11:08:052
24728996634,12cyclictest181rcu_preempt07:57:572
24728996634,12cyclictest181rcu_preempt07:57:572
24728996631,25cyclictest0-21swapper/208:14:062
24728996631,25cyclictest0-21swapper/208:14:062
24728996630,8cyclictest0-21swapper/207:45:172
24720996653,9cyclictest24697-21cyclictest10:16:091
24720996653,9cyclictest24697-21cyclictest10:16:091
24720996646,7cyclictest181rcu_preempt10:22:091
24720996642,16cyclictest27504-21fschecks_time08:47:561
24720996639,19cyclictest3261-21modprobe10:02:351
24720996639,19cyclictest3261-21modprobe10:02:351
2472099663,40cyclictest30149-21latency_hist07:17:391
2472099663,32cyclictest31-21ksoftirqd/111:25:301
24708996650,11cyclictest24697-21cyclictest07:57:110
24708996638,8cyclictest181rcu_preempt11:17:380
2470899663,44cyclictest1822-21irqstats10:52:590
2470899663,44cyclictest1822-21irqstats10:52:590
2470899663,36cyclictest15-21ksoftirqd/008:20:440
2470899663,36cyclictest15-21ksoftirqd/008:20:440
24708996628,11cyclictest0-21swapper/008:42:590
24734996523,29cyclictest14196-21chrt12:07:543
24728996545,11cyclictest298-21in:imuxsock07:57:362
24728996516,37cyclictest17164-21diskmemload09:31:552
24728996516,37cyclictest17164-21diskmemload09:31:552
24720996544,10cyclictest0-21swapper/108:18:111
24720996544,10cyclictest0-21swapper/108:18:111
2472099653,45cyclictest352-21runrttasks11:55:381
2472099653,45cyclictest352-21runrttasks11:55:381
24720996517,11cyclictest6214-21fschecks_count07:42:571
2472099651,32cyclictest31-21ksoftirqd/111:13:011
2473499648,37cyclictest0-21swapper/310:33:393
2473499648,37cyclictest0-21swapper/310:33:393
24734996422,31cyclictest10033-21chrt11:07:243
24728996443,11cyclictest0-21swapper/211:27:472
24728996443,11cyclictest0-21swapper/211:27:472
2472099645,29cyclictest31-21ksoftirqd/108:32:531
24720996444,14cyclictest11718-21/usr/sbin/munin09:23:021
24720996442,15cyclictest7712-21fschecks_time11:02:561
2472099643,42cyclictest30736-21fschecks_time08:57:571
2472099642,19cyclictest21469-21/usr/sbin/munin08:28:081
24720996416,11cyclictest301ktimers/108:07:381
24720996416,11cyclictest301ktimers/108:07:381
24720996414,43cyclictest301ktimers/109:08:271
24708996441,9cyclictest181rcu_preempt07:33:100
24728996347,9cyclictest0-21swapper/210:32:372
2472899633,37cyclictest20125-21head08:23:112
2472899633,37cyclictest20125-21head08:23:112
2472899632,40cyclictest8198-21wc07:48:082
2472099632,26cyclictest31-21ksoftirqd/108:37:551
24720996321,9cyclictest301ktimers/109:03:001
24708996321,33cyclictest7409-21sort11:02:450
24734996225,24cyclictest3669-21taskset10:56:223
24734996225,24cyclictest3669-21taskset10:56:223
24734996223,25cyclictest30116-21taskset10:46:263
24734996223,25cyclictest30116-21taskset10:46:263
24720996235,16cyclictest0-21swapper/107:23:041
24720996235,16cyclictest0-21swapper/107:23:041
312222610,5sleep20-21swapper/208:58:092
2473499613,41cyclictest16304-21chrt10:23:073
24734996126,21cyclictest7270-21chrt10:08:083
24734996125,24cyclictest28043-21chrt12:30:433
24734996125,24cyclictest15829-21taskset09:28:123
24734996125,24cyclictest15829-21taskset09:28:123
24734996125,23cyclictest21779-21taskset08:28:133
2473499604,42cyclictest5708-21taskset10:06:053
2473499604,40cyclictest1118-21chrt09:58:033
2473499604,40cyclictest1118-21chrt09:58:033
2473499603,45cyclictest10556-21taskset10:13:103
2473499603,45cyclictest10556-21taskset10:13:103
2473499603,43cyclictest19291-21taskset12:16:253
24734996030,19cyclictest15500-21chrt10:22:393
2472899602,23cyclictest31845-21diskmemload11:12:572
24734995927,19cyclictest18717-21chrt08:20:113
24734995927,19cyclictest18717-21chrt08:20:113
24734995926,20cyclictest1124-21taskset07:27:393
24734995926,20cyclictest1124-21taskset07:27:393
24734995925,21cyclictest30744-21chrt07:17:593
24734995924,23cyclictest17948-21chrt09:32:543
24734995922,26cyclictest29381-21taskset09:52:273
24734995922,25cyclictest3283-21chrt11:48:413
24734995921,25cyclictest4284-21taskset09:09:323
2473499584,37cyclictest13552-21taskset11:12:513
2473499583,42cyclictest19772-21taskset11:22:563
2473499583,40cyclictest5954-21taskset10:59:223
24734995828,19cyclictest10827-21chrt07:57:393
24734995826,20cyclictest32147-21taskset09:02:433
24734995825,21cyclictest25018-21chrt08:38:133
24734995824,21cyclictest13812-21chrt12:07:383
24734995822,24cyclictest20290-21taskset10:29:523
24734995822,23cyclictest21255-21taskset12:18:323
24734995822,23cyclictest21255-21taskset12:18:323
24734995821,24cyclictest22430-21taskset08:32:513
2473499573,35cyclictest15656-21taskset08:12:383
2473499573,35cyclictest15656-21taskset08:12:383
24734995725,22cyclictest23216-21chrt12:22:533
24734995724,21cyclictest739-21chrt10:51:403
24734995724,20cyclictest9139-21chrt11:58:143
24734995723,22cyclictest26833-21chrt11:33:543
24734995723,22cyclictest17105-21chrt11:18:063
24734995722,23cyclictest8933-21taskset09:17:553
24734995722,23cyclictest8933-21taskset09:17:553
24734995722,21cyclictest6790-21chrt07:43:103
24734995721,24cyclictest25588-21taskset09:45:143
24734995721,24cyclictest25588-21taskset09:45:143
24734995720,24cyclictest3827-21taskset07:33:343
24734995720,22cyclictest0-21swapper/307:08:053
2472899574,39cyclictest21057-21fschecks_time08:27:572
24734995624,21cyclictest13752-21chrt09:25:453
24734995624,18cyclictest29849-21chrt08:53:123
24734995623,22cyclictest27255-21chrt08:47:493
24734995623,21cyclictest23223-21chrt09:42:023
24734995622,22cyclictest19189-21taskset08:22:503
24734995622,22cyclictest19189-21taskset08:22:503
24734995622,22cyclictest15980-21chrt08:12:543
24734995622,22cyclictest15980-21chrt08:12:543
24734995619,25cyclictest1440-21taskset11:46:553
24734995619,25cyclictest1440-21taskset11:46:553
24734995524,20cyclictest10598-21chrt11:07:533
2473499552,41cyclictest25642-21taskset10:38:113
2473499552,38cyclictest5172-21chrt07:38:083
24734995523,21cyclictest5087-21chrt11:52:513
24734995523,21cyclictest5087-21chrt11:52:513
24734995523,21cyclictest28673-21chrt11:37:563
24734995523,21cyclictest11224-21chrt07:58:003
24734995523,21cyclictest11224-21chrt07:58:003
24734995523,20cyclictest31493-21chrt08:58:183
24734995522,21cyclictest8859-21chrt07:51:243
24734995522,21cyclictest29825-21chrt07:13:333
24734995425,19cyclictest25134-21chrt07:02:563
24734995424,19cyclictest6827-21chrt09:13:133
24734995422,20cyclictest13637-21chrt08:05:253
24734995422,20cyclictest13637-21chrt08:05:253
24734995420,22cyclictest26871-21chrt08:46:193
24734995321,22cyclictest23665-21chrt11:28:313
24734995321,22cyclictest23665-21chrt11:28:313
24734995321,21cyclictest1329-21chrt07:27:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional