You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-08 - 11:07
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Wed Apr 08, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
723699996,73cyclictest14616-21apt-get21:03:521
7236999719,58cyclictest19642-21apt-get19:38:551
7231999763,10cyclictest181rcu_preempt21:19:000
7231999761,8cyclictest181rcu_preempt20:53:510
723699966,38cyclictest31180-21apt-get22:24:261
7231999455,14cyclictest300-21rs:main0
7231999333,35cyclictest15-21ksoftirqd/020:46:110
7231999263,8cyclictest181rcu_preempt19:59:160
586429218,33sleep10-21swapper/118:58:581
7231999158,21cyclictest5339-21ssh22:34:370
7231999158,21cyclictest5339-21ssh22:34:370
7231998967,16cyclictest10821-21sendmail19:13:380
7242998868,14cyclictest335-21apt-get20:18:532
7242998854,25cyclictest10200-21cut20:49:012
7236998864,16cyclictest32707-21fschecks_count00:13:571
7236998864,16cyclictest32707-21fschecks_count00:13:571
7236998811,54cyclictest4392-21apt-get22:33:581
7236998811,54cyclictest4392-21apt-get22:33:581
7231998860,8cyclictest181rcu_preempt20:34:570
7231998860,8cyclictest181rcu_preempt20:34:570
7231998857,9cyclictest181rcu_preempt22:56:060
723199883,46cyclictest15-21ksoftirqd/019:30:580
7231998768,13cyclictest30279-21cstates21:28:550
7231998754,9cyclictest181rcu_preempt19:49:110
7236998665,13cyclictest0-21swapper/119:43:481
7231998655,8cyclictest181rcu_preempt00:09:050
7231998652,10cyclictest181rcu_preempt20:04:010
7236998555,10cyclictest181rcu_preempt23:03:491
7231998552,11cyclictest181rcu_preempt21:41:040
7236998462,11cyclictest0-21swapper/100:33:371
723699843,73cyclictest9974-21taskset21:48:551
7231998457,8cyclictest181rcu_preempt00:01:470
7242998365,12cyclictest365-21ntpd23:34:212
7242998361,13cyclictest30799-21latency_hist20:13:402
7242998361,13cyclictest30799-21latency_hist20:13:402
7242998320,39cyclictest39-21ksoftirqd/223:48:392
7236998359,14cyclictest32158-21munin-run00:13:371
7236998351,9cyclictest181rcu_preempt22:44:121
7231998343,7cyclictest181rcu_preempt00:32:300
7231998318,38cyclictest15-21ksoftirqd/019:44:390
7242998251,11cyclictest181rcu_preempt21:31:162
723699823,55cyclictest8781-21munin-plugin-st19:08:381
7231998267,8cyclictest14506-21ssh23:43:520
7231998263,14cyclictest5014-21latency_hist20:33:400
7231998263,14cyclictest5014-21latency_hist20:33:400
7231998243,20cyclictest181rcu_preempt21:49:210
723199823,41cyclictest15-21ksoftirqd/023:24:070
7231998230,11cyclictest181rcu_preempt22:29:230
7231998230,11cyclictest181rcu_preempt22:29:230
7242998142,12cyclictest181rcu_preempt00:27:482
7242998142,12cyclictest181rcu_preempt00:27:482
7242998130,13cyclictest181rcu_preempt22:28:462
7242998130,13cyclictest181rcu_preempt22:28:462
7236998162,13cyclictest2390-21proc_pri22:29:101
7236998162,13cyclictest2390-21proc_pri22:29:101
7236998149,14cyclictest0-21swapper/121:43:381
723699813,54cyclictest10837-21apt-get19:13:521
7231998161,9cyclictest181rcu_preempt19:08:400
7231998154,8cyclictest181rcu_preempt21:11:550
7231998153,7cyclictest181rcu_preempt20:09:090
7231998153,7cyclictest181rcu_preempt20:09:090
7231998152,8cyclictest181rcu_preempt22:24:190
723199814,30cyclictest15-21ksoftirqd/021:33:560
723199814,30cyclictest15-21ksoftirqd/021:33:560
7242998060,14cyclictest29666-21sh21:28:112
7242998060,14cyclictest29666-21sh21:28:112
7242998055,16cyclictest24835-21sh21:19:112
7231998065,9cyclictest12539-21kworker/u19:0-nfsiod23:11:220
7231998065,10cyclictest9393-21cut21:48:400
7231998053,8cyclictest181rcu_preempt23:20:000
7231998046,7cyclictest181rcu_preempt23:50:140
7242997964,8cyclictest0-21swapper/223:09:142
7242997964,8cyclictest0-21swapper/221:09:132
7242997963,8cyclictest0-21swapper/223:19:112
7242997959,9cyclictest0-21swapper/223:58:552
7242997959,14cyclictest8307-21ssh22:39:012
724299792,47cyclictest39-21ksoftirqd/222:02:322
724299791,51cyclictest39-21ksoftirqd/219:06:372
7231997941,10cyclictest181rcu_preempt23:41:550
7231997941,10cyclictest181rcu_preempt23:41:550
7231997914,22cyclictest15-21ksoftirqd/019:24:070
7231997910,42cyclictest15-21ksoftirqd/021:13:490
7231997910,42cyclictest15-21ksoftirqd/021:13:490
7242997861,11cyclictest1391-21cron22:28:372
7242997826,13cyclictest181rcu_preempt23:48:432
724299782,48cyclictest39-21ksoftirqd/220:14:562
723699783,38cyclictest31-21ksoftirqd/120:18:041
723699782,37cyclictest20899-21modprobe21:13:371
723699782,23cyclictest30706-21basename20:13:371
723699782,23cyclictest30706-21basename20:13:371
7231997863,8cyclictest0-21swapper/022:48:540
7231997857,8cyclictest181rcu_preempt22:44:140
7231997856,15cyclictest27742-21latency_hist22:18:400
7231997856,15cyclictest27742-21latency_hist22:18:400
723199783,42cyclictest15-21ksoftirqd/022:11:070
7231997832,13cyclictest181rcu_preempt23:15:100
724299778,29cyclictest39-21ksoftirqd/200:31:492
7242997755,15cyclictest24214-21sendmail19:53:382
724299772,46cyclictest39-21ksoftirqd/221:38:452
7242997714,36cyclictest39-21ksoftirqd/220:29:442
7242997714,36cyclictest39-21ksoftirqd/220:29:442
7236997751,18cyclictest13163-21sh21:53:581
7236997751,18cyclictest13163-21sh21:53:581
7236997746,23cyclictest1-21systemd22:15:191
7236997746,23cyclictest1-21systemd22:15:191
7236997744,15cyclictest0-21swapper/120:03:591
7231997756,8cyclictest181rcu_preempt00:05:080
723199775,45cyclictest15-21ksoftirqd/019:15:480
7231997751,9cyclictest181rcu_preempt22:41:210
7231997740,9cyclictest181rcu_preempt23:58:090
723199773,42cyclictest15-21ksoftirqd/022:08:400
723199773,42cyclictest15-21ksoftirqd/022:08:400
723199772,24cyclictest15-21ksoftirqd/023:05:270
100962770,9sleep10-21swapper/119:09:131
7242997657,12cyclictest17524-21idleruntime-cro19:33:382
7242997654,16cyclictest25255-21fschecks_time22:14:002
7242997654,16cyclictest25255-21fschecks_time22:14:002
7242997632,11cyclictest181rcu_preempt00:09:032
724299762,38cyclictest39-21ksoftirqd/223:40:342
724299762,38cyclictest39-21ksoftirqd/223:40:342
724299762,36cyclictest39-21ksoftirqd/223:28:312
7242997611,39cyclictest39-21ksoftirqd/221:45:492
723699762,47cyclictest12267-21ps20:54:091
723699762,34cyclictest31-21ksoftirqd/122:19:001
723699762,34cyclictest31-21ksoftirqd/122:19:001
7231997648,22cyclictest1-21systemd22:23:380
7231997648,22cyclictest1-21systemd22:23:380
7231997647,20cyclictest20976-21modprobe19:43:370
7231997645,7cyclictest181rcu_preempt21:56:370
7231997645,7cyclictest181rcu_preempt21:56:370
710127610,40sleep07108-21modprobe19:03:380
7242997555,14cyclictest6132-21unixbench_singl20:34:142
7242997555,14cyclictest6132-21unixbench_singl20:34:142
7236997561,8cyclictest31-21ksoftirqd/123:59:081
7236997552,17cyclictest3400-21idleruntime-cro21:38:381
7236997552,17cyclictest3400-21idleruntime-cro21:38:381
7231997557,9cyclictest0-21swapper/020:18:410
7231997552,10cyclictest181rcu_preempt21:59:140
723199754,46cyclictest15-21ksoftirqd/000:20:040
7242997457,8cyclictest0-21swapper/222:09:082
724299745,40cyclictest39-21ksoftirqd/220:41:252
7242997437,11cyclictest181rcu_preempt20:06:402
7242997435,10cyclictest181rcu_preempt00:15:102
7242997435,10cyclictest181rcu_preempt00:15:102
724299742,43cyclictest39-21ksoftirqd/200:04:072
724299741,39cyclictest39-21ksoftirqd/222:53:372
7242997412,30cyclictest39-21ksoftirqd/220:48:372
7236997456,11cyclictest15259-21kworker/u19:3-rpciod21:13:591
7236997456,11cyclictest15259-21kworker/u19:3-rpciod21:13:591
7236997443,9cyclictest181rcu_preempt19:37:461
7236997443,9cyclictest181rcu_preempt19:37:461
723699743,27cyclictest9378-21modprobe21:48:371
7231997460,10cyclictest7222-21cyclictest00:25:540
7231997460,10cyclictest7222-21cyclictest00:25:540
7231997455,13cyclictest18405-21processes19:34:100
7231997455,13cyclictest18405-21processes19:34:100
7231997435,8cyclictest181rcu_preempt20:23:410
723199743,46cyclictest15-21ksoftirqd/019:54:270
724299735,10cyclictest14621-21diskmemload22:33:582
724299735,10cyclictest14621-21diskmemload22:33:582
7242997339,10cyclictest181rcu_preempt20:54:152
7242997326,13cyclictest181rcu_preempt23:05:482
7236997356,11cyclictest14281-21cron19:23:381
7236997355,11cyclictest58-21kcompactd022:07:191
7236997355,11cyclictest58-21kcompactd022:07:191
7236997343,9cyclictest181rcu_preempt20:20:031
7236997342,8cyclictest3184-21ssh23:24:091
7236997332,16cyclictest181rcu_preempt20:44:021
7236997331,11cyclictest181rcu_preempt23:15:081
723699732,36cyclictest15948-21latency_hist19:28:391
7231997350,17cyclictest2871-21latency_hist00:18:400
7231997350,17cyclictest2871-21latency_hist00:18:400
7231997332,12cyclictest181rcu_preempt21:25:250
7231997332,12cyclictest181rcu_preempt21:25:250
7242997250,14cyclictest2952-21systemd-journal23:56:212
7242997243,21cyclictest12698-21grep21:53:402
724299723,50cyclictest27570-21latency_hist20:03:412
7242997232,10cyclictest181rcu_preempt19:39:522
7242997213,23cyclictest39-21ksoftirqd/220:24:042
7236997241,14cyclictest22399-21sort22:09:011
7236997239,11cyclictest181rcu_preempt23:48:371
7236997223,25cyclictest24814-21if_eth019:54:021
7231997252,8cyclictest181rcu_preempt20:24:130
7231997246,7cyclictest181rcu_preempt23:29:340
7231997246,7cyclictest181rcu_preempt19:19:520
7231997243,8cyclictest181rcu_preempt20:59:030
724299719,38cyclictest39-21ksoftirqd/222:48:332
7242997149,15cyclictest277-21dbus-daemon21:03:392
724299712,44cyclictest39-21ksoftirqd/222:57:582
724299712,29cyclictest39-21ksoftirqd/219:19:102
7242997114,32cyclictest39-21ksoftirqd/221:37:072
7242997114,32cyclictest39-21ksoftirqd/221:37:072
7236997141,10cyclictest181rcu_preempt22:58:511
7236997135,12cyclictest181rcu_preempt21:24:291
7236997135,12cyclictest181rcu_preempt21:24:291
7236997127,33cyclictest18425-21timerwakeupswit23:49:141
7231997150,8cyclictest181rcu_preempt23:34:090
7231997142,9cyclictest181rcu_preempt20:49:010
723199713,42cyclictest15-21ksoftirqd/020:40:020
7242997054,10cyclictest5242-21ssh00:22:212
7242997041,8cyclictest181rcu_preempt19:36:532
7242997041,8cyclictest181rcu_preempt19:36:532
7242997033,10cyclictest181rcu_preempt23:33:382
7242997031,10cyclictest181rcu_preempt22:19:052
7242997031,10cyclictest181rcu_preempt22:19:052
7242997013,32cyclictest39-21ksoftirqd/219:13:402
723699707,40cyclictest3703-21grep00:19:111
7236997047,8cyclictest181rcu_preempt23:34:061
7236997046,10cyclictest58-21kcompactd021:21:111
7236997045,9cyclictest31-21ksoftirqd/100:24:561
7236997045,9cyclictest31-21ksoftirqd/100:24:561
7236997022,8cyclictest0-21swapper/120:33:491
7236997022,8cyclictest0-21swapper/120:33:491
7236997020,23cyclictest27646-21latency_hist20:03:411
724299696,35cyclictest39-21ksoftirqd/221:54:162
724299696,35cyclictest39-21ksoftirqd/221:54:162
724299696,27cyclictest39-21ksoftirqd/219:23:442
7242996954,10cyclictest7222-21cyclictest23:18:402
724299694,23cyclictest39-21ksoftirqd/222:08:222
724299694,23cyclictest39-21ksoftirqd/222:08:222
7242996939,9cyclictest181rcu_preempt19:54:052
7242996935,17cyclictest58-21kcompactd021:16:422
7242996935,17cyclictest58-21kcompactd021:16:422
724299692,26cyclictest39-21ksoftirqd/221:05:182
723699694,41cyclictest8230-21latency_hist23:33:401
7236996937,12cyclictest181rcu_preempt23:54:351
723699692,39cyclictest8903-21ssh22:39:131
723199694,36cyclictest20291-21fschecks_time22:59:010
724299685,37cyclictest39-21ksoftirqd/223:02:262
724299683,56cyclictest11346-21apt-get19:14:062
7236996855,9cyclictest7222-21cyclictest21:30:411
7236996843,9cyclictest181rcu_preempt00:06:081
7236996843,19cyclictest31-21ksoftirqd/123:23:371
723699682,17cyclictest18020-21grep22:54:151
7236996811,15cyclictest23002-21taskset19:48:561
7236996740,20cyclictest16752-21sh21:59:131
7236996736,17cyclictest0-21swapper/123:41:581
7236996736,17cyclictest0-21swapper/123:41:581
7236996735,10cyclictest181rcu_preempt22:48:451
723699671,31cyclictest352-21runrttasks19:32:161
724299666,31cyclictest39-21ksoftirqd/219:43:542
7236996638,21cyclictest7599-21timerandwakeup20:39:141
7236996636,13cyclictest0-21swapper/123:12:041
7236996626,18cyclictest4294-21sort20:29:121
7236996626,18cyclictest4294-21sort20:29:121
7236996615,10cyclictest0-21swapper/120:23:521
65982658,34sleep26684-21sed18:59:242
723699643,27cyclictest10027-21apt-get20:48:511
7236996429,11cyclictest181rcu_preempt21:03:411
724899612,38cyclictest15253-21taskset21:04:013
7248995524,22cyclictest14533-21chrt21:55:333
7248995524,22cyclictest14533-21chrt21:55:333
724899543,39cyclictest20130-21taskset22:58:563
724899542,41cyclictest1991-21chrt23:23:113
724899532,40cyclictest2642-21chrt21:36:323
724899532,40cyclictest2642-21chrt21:36:323
724899523,40cyclictest5287-21taskset21:40:333
724899523,38cyclictest30232-21taskset21:28:543
724899523,38cyclictest12996-21chrt20:58:383
724899523,37cyclictest29219-21chrt23:13:593
724899523,36cyclictest18160-21taskset21:08:473
641925211,22sleep30-21swapper/318:59:213
724899513,38cyclictest17620-21taskset22:00:483
724899513,36cyclictest29756-21chrt00:08:583
724899513,36cyclictest17603-21chrt22:54:093
724899513,34cyclictest31063-21chrt22:23:413
724899513,34cyclictest31063-21chrt22:23:413
724899512,38cyclictest19140-21taskset22:03:563
724899512,38cyclictest19140-21taskset22:03:563
7248995035,9cyclictest144482sleep319:23:413
724899503,37cyclictest7714-21chrt19:04:003
724899503,37cyclictest27016-21chrt19:59:163
724899503,36cyclictest27112-21taskset22:17:093
724899503,36cyclictest27112-21taskset22:17:093
724899503,36cyclictest10250-21chrt21:49:413
724899503,35cyclictest31755-21chrt22:24:553
724899503,35cyclictest14797-21taskset21:03:403
724899503,34cyclictest11468-21chrt23:38:473
724899503,34cyclictest11468-21chrt23:38:473
724899502,37cyclictest24399-21chrt19:53:413
724899493,36cyclictest15331-21chrt19:24:133
724899493,36cyclictest10743-21chrt00:30:593
724899493,35cyclictest26857-21chrt00:04:003
724899493,35cyclictest25773-21chrt21:20:393
724899493,34cyclictest9145-21chrt23:34:093
724899493,34cyclictest25318-21chrt19:54:143
724899493,34cyclictest2044-21chrt20:23:553
724899493,34cyclictest1380-21chrt20:20:223
724899493,33cyclictest8324-21taskset23:33:413
724899493,33cyclictest28781-21chrt20:05:203
724899492,36cyclictest5043-21chrt00:21:473
724899492,35cyclictest7998-21chrt22:38:543
724899484,31cyclictest25457-21taskset00:01:413
724899483,34cyclictest8510-21chrt20:43:553
724899483,34cyclictest5403-21chrt20:33:593
724899483,34cyclictest5403-21chrt20:33:593
724899483,34cyclictest3879-21chrt20:29:013
724899483,34cyclictest3879-21chrt20:29:013
724899483,34cyclictest23940-21taskset21:18:403
724899483,34cyclictest23940-21taskset21:18:403
724899483,34cyclictest21410-21chrt19:43:573
724899483,34cyclictest20706-21chrt23:53:593
724899483,33cyclictest23159-21chrt23:03:583
724899482,34cyclictest8294-21taskset21:45:363
724899482,34cyclictest7244-21chrt22:37:323
724899482,34cyclictest7244-21chrt22:37:323
724899473,35cyclictest13016-21chrt22:46:563
724899473,34cyclictest28231-21chrt23:12:333
724899473,32cyclictest3804-21chrt22:32:003
724899473,32cyclictest3804-21chrt22:32:003
724899473,30cyclictest32710-21chrt00:13:573
724899473,30cyclictest32710-21chrt00:13:573
724899473,30cyclictest18664-21chrt19:34:133
724899473,30cyclictest18664-21chrt19:34:133
724899463,31cyclictest14373-21chrt22:49:033
724899462,31cyclictest11322-21taskset19:13:443
724899453,33cyclictest18316-21chrt23:49:133
724899453,29cyclictest6724-21chrt00:24:103
724899453,29cyclictest6724-21chrt00:24:103
724899433,31cyclictest9619-21taskset19:08:593
724899433,27cyclictest27932-21chrt21:24:123
724899433,27cyclictest27932-21chrt21:24:123
724899423,28cyclictest31878-21chrt20:14:153
724899423,28cyclictest29324-21taskset20:08:523
724899423,28cyclictest29324-21taskset20:08:523
724899422,29cyclictest4949-21chrt23:28:053
724899422,28cyclictest21144-21taskset19:43:413
724899422,27cyclictest16773-21chrt23:47:353
724899412,30cyclictest10034-21chrt20:48:543
724899412,29cyclictest6809-21taskset20:38:493
724899392,28cyclictest23228-21taskset22:09:433
724899392,27cyclictest16296-21chrt19:28:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional