You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 07:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Wed May 13, 2026 00:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3241121620,23sleep20-21swapper/218:57:372
32220212412,82sleep132175-21apt-get18:57:041
3252021159,66sleep032155-21/usr/sbin/munin18:57:520
1202991157,64cyclictest7388-21ssh22:47:181
12099911451,18cyclictest1176-21cyclictest23:06:192
1209991136,58cyclictest6544-21apt-get22:46:392
118899985,75cyclictest31571-21apt-get23:27:260
120999967,51cyclictest31226-21apt-get23:26:352
120999947,41cyclictest8682-21ssh21:01:362
118899921,41cyclictest0-21swapper/000:21:550
1202999148,9cyclictest181rcu_preempt00:06:331
1188999029,40cyclictest0-21swapper/022:31:550
1188999029,40cyclictest0-21swapper/022:31:550
120999893,33cyclictest352-21runrttasks20:50:192
118899897,73cyclictest22156-21apt-get23:11:330
118899897,73cyclictest22156-21apt-get23:11:330
1209998866,15cyclictest1176-21cyclictest00:21:182
120299885,60cyclictest8646-21apt-get21:01:381
118899883,61cyclictest14883-21apt-get22:06:310
1209998770,11cyclictest27771-21systemctl23:21:172
120999873,58cyclictest15426-21apt-get22:06:312
1202998767,14cyclictest58-21kcompactd022:56:421
1188998759,20cyclictest2460-21processes20:41:500
1209998664,15cyclictest1-21systemd22:39:052
1209998627,12cyclictest181rcu_preempt21:46:202
118899865,47cyclictest17487-21timerwakeupswit23:56:530
121499853,31cyclictest15504-21chrt22:06:393
1188998560,16cyclictest8081-21diskmemload22:26:560
1188998558,17cyclictest30145-21mailstats20:26:490
1188998558,17cyclictest30145-21mailstats20:26:490
1188998556,20cyclictest16381-21latency_hist19:46:190
1188998555,19cyclictest24089-21latency_hist22:21:200
1188998537,23cyclictest0-21swapper/021:02:200
120299847,66cyclictest6479-21ssh21:52:021
1202998459,17cyclictest4443-21apt-key00:31:181
1202998459,17cyclictest4443-21apt-key00:31:181
1188998425,33cyclictest0-21swapper/000:21:170
1209998363,15cyclictest1176-21cyclictest21:32:042
120999833,51cyclictest22156-21apt-get23:11:312
120999833,51cyclictest22156-21apt-get23:11:312
1202998361,16cyclictest16489-21latency_hist19:46:201
1202998353,8cyclictest181rcu_preempt22:07:241
1202998349,8cyclictest181rcu_preempt00:11:541
1202998311,57cyclictest22840-21tune2fs23:12:571
1202998311,57cyclictest22840-21tune2fs23:12:571
118899831,31cyclictest0-21swapper/020:36:320
118899831,31cyclictest0-21swapper/020:36:320
1209998255,13cyclictest181rcu_preempt22:13:372
1202998251,14cyclictest20212-21ssh21:21:071
1202998239,8cyclictest181rcu_preempt23:46:191
1188998264,12cyclictest277-21dbus-daemon23:36:180
1188998264,12cyclictest277-21dbus-daemon23:36:180
1209998164,11cyclictest503-21sh22:36:082
1209998164,11cyclictest503-21sh22:36:082
1209998160,14cyclictest28421-21kworker/u18:0-flush-179:020:01:402
1202998151,11cyclictest181rcu_preempt00:21:451
1202998149,14cyclictest5004-21ssh23:36:391
1202998137,31cyclictest6396-21idleruntime-cro19:16:171
1202998125,41cyclictest5891-21timerandwakeup20:51:531
1188998124,35cyclictest0-21swapper/020:06:340
118899812,36cyclictest0-21swapper/022:56:190
1188998120,34cyclictest0-21swapper/022:45:570
1188998112,39cyclictest30676-21apt-get23:26:160
1209998058,17cyclictest21619-21kworker/u18:2+events_unbound20:01:172
1209998058,17cyclictest21619-21kworker/u18:2+events_unbound20:01:172
1209998053,21cyclictest3308-21fschecks_count21:46:392
1202998045,10cyclictest21821-21ssh00:05:321
1202998045,10cyclictest21821-21ssh00:05:321
1202998041,14cyclictest25920-21ssh23:16:531
1188998059,12cyclictest26348-21ssh21:31:200
1188998048,11cyclictest29274-21sh21:36:180
1188998031,35cyclictest7094-21apt-get22:46:340
1188998030,25cyclictest0-21swapper/021:51:180
1188998012,53cyclictest5714-21apt-get21:51:470
120299796,53cyclictest31226-21apt-get23:26:311
118899793,34cyclictest19483-21apt-get00:01:350
118899793,34cyclictest19483-21apt-get00:01:350
121499783,34cyclictest31823-21chrt00:21:453
1209997852,8cyclictest181rcu_preempt21:54:032
1202997855,15cyclictest8081-21diskmemload21:56:441
1202997828,12cyclictest181rcu_preempt22:26:201
1188997813,32cyclictest0-21swapper/022:37:200
121499774,51cyclictest0-21swapper/323:11:313
121499774,51cyclictest0-21swapper/323:11:313
1209997760,11cyclictest0-21swapper/200:01:192
1209997759,12cyclictest26046-21ssh21:30:462
1209997748,11cyclictest181rcu_preempt22:26:192
1209997747,8cyclictest181rcu_preempt22:41:552
1209997746,21cyclictest19532-21modprobe19:56:162
1209997746,21cyclictest19532-21modprobe19:56:162
120299773,38cyclictest32494-21wc20:36:181
120299773,38cyclictest32494-21wc20:36:181
1188997759,9cyclictest0-21swapper/021:36:340
1188997759,9cyclictest0-21swapper/021:36:340
1188997759,13cyclictest10739-21kworker/u17:0+xprtiod00:11:340
1188997725,28cyclictest8081-21diskmemload23:46:180
121499763,33cyclictest22448-21chrt00:06:333
121499762,31cyclictest10101-21taskset19:26:283
1209997659,11cyclictest8081-21diskmemload23:06:392
120999762,25cyclictest15595-21timerwakeupswit21:11:522
1202997640,8cyclictest181rcu_preempt21:21:291
1202997638,10cyclictest181rcu_preempt19:36:411
1202997626,17cyclictest181rcu_preempt21:29:121
1188997658,12cyclictest1176-21cyclictest23:02:350
118899765,31cyclictest0-21swapper/021:26:190
1188997649,21cyclictest14285-21ssh23:51:530
118899763,44cyclictest13210-21latency_hist23:51:210
1188997622,23cyclictest0-21swapper/022:11:460
120999758,57cyclictest6400-21apt-get19:16:312
120999758,57cyclictest6400-21apt-get19:16:312
1209997557,12cyclictest6491-21true20:56:182
1209997540,14cyclictest181rcu_preempt20:13:312
1202997554,16cyclictest1176-21cyclictest22:40:221
1202997536,11cyclictest181rcu_preempt20:21:501
1202997527,8cyclictest16033-21ssh23:01:341
118899752,31cyclictest11905-21modprobe22:01:160
118899751,44cyclictest23214-21switchtime00:06:520
118899751,33cyclictest0-21swapper/021:11:320
1188997511,26cyclictest0-21swapper/021:41:430
1209997452,16cyclictest1176-21cyclictest00:03:402
1209997452,16cyclictest1176-21cyclictest00:03:402
1209997445,8cyclictest0-21swapper/223:32:352
1209997445,8cyclictest0-21swapper/223:32:352
1209997444,9cyclictest181rcu_preempt20:56:332
1209997441,9cyclictest181rcu_preempt20:37:192
1209997441,9cyclictest181rcu_preempt20:37:192
1202997458,10cyclictest1176-21cyclictest23:31:511
1202997458,10cyclictest1176-21cyclictest23:31:511
1202997432,29cyclictest3803-21grep19:06:451
1188997449,19cyclictest8081-21diskmemload00:26:350
1188997449,19cyclictest8081-21diskmemload00:26:350
1188997414,28cyclictest0-21swapper/019:22:100
121499734,27cyclictest21489-21chrt20:01:353
121499733,29cyclictest18063-21taskset23:04:423
121499733,25cyclictest593-21taskset22:36:173
121499733,25cyclictest593-21taskset22:36:173
1209997350,15cyclictest31542-21fschecks_time00:21:402
1209997338,9cyclictest181rcu_preempt19:36:382
1202997355,11cyclictest11244-21sh23:46:491
1202997351,16cyclictest21708-21apt-key23:11:181
1202997342,22cyclictest17601-21grep23:56:541
1202997330,19cyclictest22079-21mailstats20:01:521
1188997351,13cyclictest27742-21ssh23:21:160
1188997349,13cyclictest0-21swapper/020:18:370
1188997344,22cyclictest0-21swapper/023:36:310
118899732,38cyclictest18943-21apt-get23:06:290
121499723,33cyclictest21222-21taskset22:16:193
1209997252,14cyclictest2374-21systemd-journal22:59:082
1209997252,13cyclictest1176-21cyclictest22:16:532
1209997250,12cyclictest0-21swapper/219:01:402
1209997235,18cyclictest0-21swapper/222:29:082
1202997246,20cyclictest14308-21latency_hist21:11:211
1202997242,10cyclictest26871-21/usr/sbin/munin20:16:511
1202997237,8cyclictest181rcu_preempt19:56:331
1202997237,8cyclictest181rcu_preempt19:56:331
1202997223,17cyclictest4889-21kworker/u18:1+rpciod21:51:211
1202997217,28cyclictest30848-21sendmail20:31:171
1202997217,28cyclictest30848-21sendmail20:31:171
118899725,45cyclictest6880-21/usr/sbin/munin19:17:390
118899725,45cyclictest6880-21/usr/sbin/munin19:17:390
118899724,32cyclictest12899-21grep22:01:490
1188997216,23cyclictest26197-21rm22:23:520
1188997212,20cyclictest0-21swapper/019:11:430
46327122,34sleep30-21swapper/318:58:083
121499712,34cyclictest9979-21chrt19:26:203
1209997149,14cyclictest29568-21sh23:23:062
1209997136,11cyclictest181rcu_preempt22:00:352
120999713,29cyclictest8787-21cut19:21:582
1202997155,10cyclictest11557-21latency_hist19:31:191
1202997150,15cyclictest1024-21timerwakeupswit21:41:531
1202997148,16cyclictest1176-21cyclictest19:06:111
1202997136,8cyclictest181rcu_preempt20:36:311
1202997136,8cyclictest181rcu_preempt20:36:311
118899714,32cyclictest10094-21/usr/sbin/munin19:26:440
118899713,50cyclictest28782-21unixbench_multi20:21:520
1188997130,31cyclictest0-21swapper/019:56:420
1188997130,31cyclictest0-21swapper/019:56:420
1188997110,25cyclictest0-21swapper/020:51:390
121499702,33cyclictest0-21swapper/323:26:473
1209997050,14cyclictest23952-21unixbench_singl20:06:532
1209997047,16cyclictest7355-21systemd-run23:41:182
1209997043,19cyclictest14943-21/usr/sbin/munin19:41:462
1209997039,11cyclictest181rcu_preempt00:28:032
1209997039,11cyclictest181rcu_preempt00:28:032
1209997028,13cyclictest181rcu_preempt00:07:112
1202997056,8cyclictest0-21swapper/119:46:531
118899704,35cyclictest8461-21latency_hist21:01:220
118899701,32cyclictest15783-21cat23:01:190
121499693,46cyclictest0-21swapper/322:46:323
121499692,27cyclictest2050-21chrt00:26:393
121499692,27cyclictest2050-21chrt00:26:393
1209996950,12cyclictest0-21swapper/222:51:542
1209996944,19cyclictest14572-21unixbench_multi23:51:562
1209996940,11cyclictest181rcu_preempt21:18:422
1209996935,12cyclictest181rcu_preempt23:42:322
120999693,47cyclictest18109-21latency_hist19:51:212
1209996933,13cyclictest181rcu_preempt23:47:582
1202996947,16cyclictest13183-21timerwakeupswit22:01:531
1202996947,13cyclictest0-21swapper/122:54:351
1202996945,13cyclictest0-21swapper/121:36:461
1202996945,13cyclictest0-21swapper/121:36:461
1202996943,8cyclictest181rcu_preempt22:33:091
1202996943,8cyclictest181rcu_preempt22:33:091
1202996941,20cyclictest30582-21sh23:26:021
1202996939,12cyclictest1176-21cyclictest20:41:521
120299693,27cyclictest0-21swapper/100:18:441
1202996925,9cyclictest181rcu_preempt20:06:391
118899693,36cyclictest17612-21/usr/sbin/munin21:16:550
118899693,32cyclictest17896-21modprobe19:51:170
118899692,37cyclictest31115-21apt-get20:31:300
118899692,37cyclictest31115-21apt-get20:31:300
1209996818,9cyclictest0-21swapper/220:41:302
1209996818,24cyclictest31464-21uname20:31:442
1209996818,24cyclictest31464-21uname20:31:442
1202996854,10cyclictest1176-21cyclictest19:32:351
1202996852,10cyclictest4060-21ssh22:41:361
1202996843,16cyclictest8348-21/usr/sbin/munin19:22:051
1202996835,22cyclictest15312-21wc21:11:481
1202996811,12cyclictest31-21ksoftirqd/121:31:421
1188996831,28cyclictest0-21swapper/019:34:020
118899681,30cyclictest3689-21grep20:46:380
118899681,27cyclictest21576-21diskstats20:01:390
1188996811,22cyclictest25243-21processes20:11:490
121499673,25cyclictest9293-21chrt23:43:323
1209996744,17cyclictest14311-21idleruntime-cro21:11:172
1209996734,18cyclictest29937-21/usr/sbin/munin21:36:412
1209996734,18cyclictest29937-21/usr/sbin/munin21:36:412
1209996732,19cyclictest0-21swapper/219:11:452
1202996750,12cyclictest1176-21cyclictest22:20:291
1202996745,12cyclictest0-21swapper/119:51:521
1202996745,12cyclictest0-21swapper/119:51:521
120299673,46cyclictest16299-21latency_hist23:56:181
120299673,45cyclictest27533-21runrttasks22:26:351
118899672,30cyclictest18195-21apt-get19:51:320
118899672,30cyclictest18195-21apt-get19:51:320
1209996646,11cyclictest0-21swapper/219:31:322
1209996635,17cyclictest181rcu_preempt19:26:522
120299662,48cyclictest24848-21fschecks_count20:11:381
1188996648,11cyclictest300-21rs:main0
118899661,30cyclictest0-21swapper/019:36:510
121499654,47cyclictest6930-21chrt19:16:483
121499654,47cyclictest6930-21chrt19:16:483
1209996548,10cyclictest0-21swapper/220:16:412
1209996544,8cyclictest181rcu_preempt19:06:232
1209996543,9cyclictest181rcu_preempt00:13:092
1209996535,11cyclictest0-21swapper/221:25:252
1209996534,8cyclictest181rcu_preempt20:21:312
120299655,38cyclictest352-21runrttasks21:00:371
120299655,23cyclictest297-21systemd-logind19:21:181
120299655,23cyclictest297-21systemd-logind19:21:181
118899654,34cyclictest12075-21ssh21:06:470
118899651,30cyclictest1893-21awk19:01:460
1209996446,12cyclictest14914-21apt-key22:06:182
1202996447,11cyclictest277-21dbus-daemon22:16:191
120299643,40cyclictest5002-21latency_hist20:51:201
1209996324,10cyclictest0-21swapper/220:31:212
120999623,43cyclictest29488-21cpu20:26:322
121499584,37cyclictest3693-21taskset20:46:393
121499574,33cyclictest23975-21taskset20:06:543
121499573,38cyclictest3607-21taskset21:46:453
121499563,36cyclictest13682-21taskset23:51:343
121499553,39cyclictest21418-21taskset21:21:533
121499553,38cyclictest15463-21taskset23:00:403
121499553,37cyclictest5621-21taskset22:43:433
1214995510,12cyclictest0-21swapper/321:51:333
121499534,36cyclictest18590-21chrt23:59:413
121499533,38cyclictest5022-21taskset19:11:213
121499533,35cyclictest12067-21chrt19:31:383
121499532,40cyclictest5563-21taskset19:11:463
121499532,34cyclictest15149-21taskset19:41:383
121499532,33cyclictest19856-21chrt19:56:333
121499532,33cyclictest19856-21chrt19:56:333
121499523,36cyclictest14374-21chrt21:11:193
121499523,35cyclictest11540-21taskset21:06:203
121499523,34cyclictest27954-21taskset21:32:393
121499522,35cyclictest29719-21chrt22:30:263
121499522,31cyclictest672-21taskset20:36:423
121499522,31cyclictest672-21taskset20:36:423
121499514,30cyclictest29875-21chrt21:36:403
121499514,30cyclictest29875-21chrt21:36:403
121499513,36cyclictest29453-21taskset00:16:593
121499513,36cyclictest18966-21chrt19:51:513
121499513,36cyclictest18966-21chrt19:51:513
121499513,32cyclictest25468-21chrt23:16:413
121499513,32cyclictest13170-21chrt22:01:533
121499512,34cyclictest26744-21chrt20:16:463
121499512,34cyclictest20429-21chrt23:08:023
121499503,36cyclictest20219-21chrt21:21:083
121499503,35cyclictest3723-21chrt22:41:203
121499503,35cyclictest30223-21chrt20:26:503
121499503,35cyclictest30223-21chrt20:26:503
121499503,35cyclictest14325-21taskset19:36:533
121499503,34cyclictest5572-21chrt20:51:453
121499503,34cyclictest28921-21chrt23:21:523
121499502,38cyclictest22330-21chrt22:16:523
121499502,36cyclictest20860-21chrt00:02:453
121499502,36cyclictest20860-21chrt00:02:453
121499502,35cyclictest6423-21taskset23:38:423
121499502,35cyclictest25798-21chrt21:30:043
121499502,34cyclictest25855-21chrt20:14:353
121499494,34cyclictest26877-21chrt22:25:483
121499483,33cyclictest10032-21chrt21:56:503
121499483,30cyclictest2996-21chrt20:43:353
121499482,36cyclictest10382-21chrt22:51:413
121499482,33cyclictest14674-21chrt21:11:313
121499482,31cyclictest28792-21taskset20:21:523
121499473,30cyclictest1544-21taskset21:42:563
121499472,33cyclictest12110-21taskset23:48:103
121499463,33cyclictest26737-21chrt00:12:373
121499463,31cyclictest17505-21chrt19:46:533
121499453,29cyclictest2301-21chrt23:31:453
121499453,29cyclictest2301-21chrt23:31:453
121499413,26cyclictest31120-21taskset20:31:323
121499413,26cyclictest31120-21taskset20:31:323
121499402,27cyclictest6736-21chrt20:56:313
121499383,23cyclictest1353-21chrt19:01:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional