You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-11 - 07:29
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Mon May 11, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22457213013,86sleep222472-21latency_hist19:01:252
226299910630,44cyclictest352-21runrttasks21:49:330
226299910546,24cyclictest0-21swapper/000:06:410
226299910535,27cyclictest7180-21rm00:00:070
226299910534,29cyclictest29633-21ssh22:48:460
226299910523,42cyclictest1913-21latency22:56:520
226299910439,24cyclictest0-21swapper/021:11:360
226299910438,27cyclictest32455-21sh21:06:160
226349910334,11cyclictest0-21swapper/119:41:351
226299910337,26cyclictest1782-21ssh21:07:530
226299910336,28cyclictest24156-21rm22:40:080
226299910336,28cyclictest24156-21rm22:40:080
226299910330,38cyclictest30967-21munin-run23:46:240
22634991023,72cyclictest2100-21ssh23:51:421
22634991023,72cyclictest2100-21ssh23:51:421
226299910236,28cyclictest14518-21rm23:17:050
226299910235,28cyclictest14854-21rm22:24:110
226299910235,28cyclictest14854-21rm22:24:110
226299910234,30cyclictest27494-21sh21:51:310
22634991012,58cyclictest27544-21apt-get21:51:421
226299910156,38cyclictest0-21swapper/019:01:290
226299910133,26cyclictest8553-21runrttasks19:56:400
226299910132,29cyclictest3575-21ssh22:06:040
226299910132,29cyclictest3575-21ssh22:06:040
226429910080,14cyclictest2374-21systemd-journal21:09:352
226299910035,27cyclictest17646-21sh23:22:310
226299910035,26cyclictest24034-21ssh00:27:200
226299910035,26cyclictest24034-21ssh00:27:200
226299910035,26cyclictest13919-21rm00:11:310
226299910035,26cyclictest13919-21rm00:11:310
226299910034,25cyclictest22221-21ssh00:25:370
22629999939,29cyclictest8188-21ssh00:01:460
22629999939,29cyclictest8188-21ssh00:01:460
22629999938,24cyclictest31341-21ssh22:51:510
22629999938,22cyclictest12341-21/usr/sbin/munin21:26:440
22629999936,27cyclictest5221-21swap23:02:000
22629999936,27cyclictest5221-21swap23:02:000
22629999933,33cyclictest352-21runrttasks19:18:340
22629999933,33cyclictest352-21runrttasks19:18:340
22629999933,28cyclictest17180-21rm21:34:050
22629999933,27cyclictest9132-21ssh22:14:500
22629999933,27cyclictest9132-21ssh22:14:500
22629999933,27cyclictest26918-21sh23:38:250
22629999838,19cyclictest11090-21ps23:11:580
22629999838,19cyclictest11090-21ps23:11:580
22629999834,41cyclictest22749-21seq21:42:570
22629999834,24cyclictest17983-21rm22:29:360
22629999736,26cyclictest16795-21ls20:21:280
22629999735,20cyclictest26482-21taskset19:11:390
22629999735,20cyclictest26482-21taskset19:11:390
22629999734,24cyclictest3577-21ssh23:53:170
22629999734,24cyclictest3577-21ssh23:53:170
22629999733,45cyclictest352-21runrttasks22:16:400
22629999636,21cyclictest19521-21seq00:21:220
22629999633,24cyclictest8253-21taskset19:56:170
22629999632,21cyclictest14937-21seq20:15:590
22629999632,21cyclictest14937-21seq20:15:590
22629999539,37cyclictest9561-21seq21:21:450
22629999537,18cyclictest29668-21/usr/sbin/munin19:21:550
22629999537,18cyclictest29668-21/usr/sbin/munin19:21:550
22629999530,24cyclictest352-21runrttasks19:47:530
22634999458,11cyclictest181rcu_preempt19:49:441
22629999439,20cyclictest23770-21seq20:41:510
22629999435,21cyclictest19912-21rm21:38:230
22629999432,26cyclictest9196-21ssh23:08:580
22629999427,25cyclictest0-21swapper/021:20:340
22629999338,18cyclictest22885-21/usr/sbin/munin19:01:420
22629999335,22cyclictest4437-21ssh22:06:500
22629999333,21cyclictest4966-21seq19:45:470
22634999215,46cyclictest13641-21apt-get20:11:411
22634999215,46cyclictest13641-21apt-get20:11:411
22629999235,26cyclictest0-21swapper/023:46:400
22629999235,26cyclictest0-21swapper/023:46:400
22629999235,25cyclictest20256-21runrttasks20:31:450
22629999233,27cyclictest21744-21sh23:30:410
22629999233,21cyclictest18129-21taskset20:25:420
22629999233,19cyclictest10296-21df20:01:430
22634999150,17cyclictest181rcu_preempt23:01:271
22634999144,9cyclictest0-21swapper/119:06:431
22634999144,9cyclictest0-21swapper/119:06:431
22629999136,20cyclictest27084-21ssh22:44:590
22629999134,27cyclictest31993-21sh21:58:050
22629999134,25cyclictest26909-21taskset20:51:490
22629999133,24cyclictest24640-21ssh23:35:220
22629999034,21cyclictest25909-21seq20:47:020
22629999034,21cyclictest25909-21seq20:47:020
22634998957,12cyclictest181rcu_preempt19:16:421
22634998957,12cyclictest181rcu_preempt19:16:421
22629998933,20cyclictest29486-21taskset21:01:120
22629998930,23cyclictest29524-21diskmemload22:33:380
22634998873,10cyclictest7418-21unixbench_multi21:17:011
22634998859,20cyclictest14036-21grep22:22:031
22634998859,20cyclictest14036-21grep22:22:031
22629998830,27cyclictest0-21swapper/020:26:400
22629998829,22cyclictest0-21swapper/019:31:590
22629998829,22cyclictest0-21swapper/019:31:590
22634998731,19cyclictest0-21swapper/123:26:411
22629998735,17cyclictest352-21runrttasks19:26:460
22629998732,20cyclictest12829-21seq20:08:060
22629998719,28cyclictest0-21swapper/019:06:580
22629998719,28cyclictest0-21swapper/019:06:580
22634998667,13cyclictest16914-21apt-get00:16:361
22634998648,16cyclictest181rcu_preempt21:46:201
22629998634,22cyclictest19676-21snmpd20:38:370
22642998567,10cyclictest0-21swapper/221:11:362
22642998561,17cyclictest13270-21sh23:16:282
22642998561,17cyclictest13270-21sh23:16:282
22642998559,18cyclictest28496-21latency19:16:532
22642998559,18cyclictest28496-21latency19:16:532
22642998552,26cyclictest29406-21kworker/u19:2+nfsiod23:56:482
22634998563,17cyclictest20057-21kworker/u19:0+events_unbound22:41:251
22634998563,17cyclictest20057-21kworker/u19:0+events_unbound22:41:251
22634998461,15cyclictest1546-40mandb23:51:261
22634998461,15cyclictest1546-40mandb23:51:261
2263499843,59cyclictest16591-21sh20:21:251
22634998434,12cyclictest181rcu_preempt21:06:491
2264299833,45cyclictest27602-21ssh21:51:442
22634998362,15cyclictest544-21cron20:51:251
22634998362,15cyclictest544-21cron20:51:251
22629998335,16cyclictest3205-21seq19:38:470
2264299829,32cyclictest30928-21apt-get22:51:382
22642998260,15cyclictest8500-21latency_hist19:56:292
22642998260,15cyclictest29524-21diskmemload23:42:012
2264299823,55cyclictest58-21kcompactd023:51:442
2264299823,55cyclictest58-21kcompactd023:51:442
22642998160,15cyclictest30251-21kernelversion21:01:492
22634998157,15cyclictest5157-21mailstats23:01:591
22634998157,15cyclictest5157-21mailstats23:01:591
22634998123,34cyclictest0-21swapper/122:41:371
215952819,40sleep10-21swapper/118:57:001
22642998062,10cyclictest0-21swapper/223:16:552
22642997935,12cyclictest0-21swapper/223:01:452
22642997935,12cyclictest0-21swapper/223:01:452
2264799783,51cyclictest0-21swapper/321:51:493
22642997853,17cyclictest6862-21latency_hist19:51:282
22642997853,16cyclictest2374-21systemd-journal23:30:342
2264299782,48cyclictest13641-21apt-get20:11:522
2264299782,48cyclictest13641-21apt-get20:11:522
22642997823,18cyclictest2374-21systemd-journal23:33:352
22634997852,17cyclictest22622-21cyclictest22:06:481
22634997852,11cyclictest181rcu_preempt22:01:111
22634997850,10cyclictest26096-21ssh21:49:001
22634997836,12cyclictest181rcu_preempt22:46:591
2264299776,36cyclictest352-21runrttasks00:17:132
22642997758,13cyclictest25743-21latency_hist00:31:292
22642997758,12cyclictest20727-21mailstats20:31:582
22642997758,10cyclictest0-21swapper/222:06:582
22634997746,8cyclictest21509-21ssh00:23:351
22634997740,12cyclictest181rcu_preempt21:22:461
22642997659,10cyclictest352-21runrttasks20:21:412
22642997659,10cyclictest24889-21sh00:29:472
22642997658,11cyclictest277-21dbus-daemon00:26:262
22642997654,16cyclictest26969-21latency_hist21:51:282
22642997640,8cyclictest181rcu_preempt23:47:002
22642997640,8cyclictest181rcu_preempt23:47:002
22634997656,12cyclictest24135-21latency_hist19:06:291
22634997650,19cyclictest544-21cron19:26:251
22634997650,19cyclictest544-21cron19:26:251
2263499763,49cyclictest16142-21ssh21:32:001
22634997549,17cyclictest32139-21sh21:05:231
22634997545,8cyclictest24840-21ssh00:29:381
22634997545,8cyclictest24840-21ssh00:29:381
22634997541,12cyclictest181rcu_preempt00:06:011
22634997541,12cyclictest181rcu_preempt00:06:011
22634997533,17cyclictest22622-21cyclictest21:41:281
2263499753,30cyclictest11599-21ntp_states00:06:561
22634997516,43cyclictest7039-21sort22:11:371
22634997516,43cyclictest7039-21sort22:11:371
2264299749,27cyclictest0-21swapper/221:22:062
22642997455,9cyclictest0-21swapper/219:11:282
22642997455,9cyclictest0-21swapper/219:11:282
22642997453,12cyclictest0-21swapper/223:26:272
22642997446,21cyclictest29631-21sh21:01:282
22642997446,21cyclictest24673-21cut20:46:272
22634997451,16cyclictest29524-21diskmemload23:45:051
22634997445,13cyclictest18350-21ssh22:30:381
22634997433,13cyclictest181rcu_preempt23:06:551
2264799734,40cyclictest0-21swapper/323:51:263
2264799734,40cyclictest0-21swapper/323:51:263
22642997354,12cyclictest0-21swapper/219:57:002
22642997351,12cyclictest0-21swapper/222:57:012
22642997318,8cyclictest92272ls21:21:282
22642997316,18cyclictest29080-21sh22:47:132
22634997351,12cyclictest0-21swapper/120:31:471
22634997347,9cyclictest32171-21ssh22:52:251
22642997252,12cyclictest10690-21systemd-run00:06:262
22642997252,12cyclictest10690-21systemd-run00:06:262
22634997248,8cyclictest19676-21snmpd19:27:021
22634997244,9cyclictest181rcu_preempt23:12:011
22634997244,9cyclictest181rcu_preempt23:12:011
2263499723,25cyclictest1426-21ssh22:01:481
2263499723,25cyclictest1426-21ssh22:01:481
22634997229,15cyclictest181rcu_preempt23:32:291
22634997228,12cyclictest181rcu_preempt19:31:431
22634997228,12cyclictest181rcu_preempt19:31:431
22634997212,39cyclictest31-21ksoftirqd/123:25:461
2264299719,23cyclictest20831-21seq21:40:582
22642997127,33cyclictest4009-21if_eth019:41:472
22642997112,22cyclictest0-21swapper/221:58:232
22642997111,27cyclictest16423-21seq20:19:232
22634997140,14cyclictest181rcu_preempt23:57:021
2263499712,48cyclictest11706-21runrttasks20:06:291
22647997016,33cyclictest0-21swapper/323:51:423
22647997016,33cyclictest0-21swapper/323:51:423
22642997049,12cyclictest0-21swapper/219:41:282
22642997041,22cyclictest19977-21latency_hist20:31:282
22642997012,24cyclictest0-21swapper/222:02:552
22642997012,24cyclictest0-21swapper/222:02:552
22642997012,23cyclictest352-21runrttasks20:51:462
22642997011,25cyclictest30862-21seq19:24:302
22642997011,25cyclictest30862-21seq19:24:302
22634997047,17cyclictest27818-21munin-plugin-st19:16:271
22634997047,17cyclictest27818-21munin-plugin-st19:16:271
22634997043,18cyclictest352-21runrttasks20:26:511
22634997042,10cyclictest181rcu_preempt20:54:411
22634997037,13cyclictest181rcu_preempt00:11:581
22634997037,13cyclictest181rcu_preempt00:11:581
22634997025,12cyclictest181rcu_preempt21:12:521
22642996949,10cyclictest0-21swapper/219:16:282
22642996949,10cyclictest0-21swapper/219:16:282
22642996947,15cyclictest277-21dbus-daemon22:31:252
22642996911,26cyclictest11852-21seq22:19:072
22634996941,19cyclictest12839-21latency21:26:521
2263499693,42cyclictest3428-21idleruntime-cro19:41:251
22634996933,7cyclictest181rcu_preempt19:56:291
2263499693,27cyclictest31-21ksoftirqd/122:17:181
2263499693,25cyclictest23167-21latency_hist20:41:281
22634996922,8cyclictest181rcu_preempt22:31:581
2264299689,29cyclictest20140-21seq22:32:122
2264299689,28cyclictest352-21runrttasks19:26:512
2264299687,26cyclictest1491-21seq19:32:272
2264299687,26cyclictest1491-21seq19:32:272
22642996847,11cyclictest15794-21latency_hist22:26:282
22642996847,11cyclictest15794-21latency_hist22:26:282
22642996846,14cyclictest21307-21acpi21:41:362
22642996845,11cyclictest0-21swapper/221:26:562
22642996810,24cyclictest22805-21seq20:38:232
2263499684,45cyclictest352-21runrttasks19:56:411
2263499683,40cyclictest29676-21latency_hist21:01:281
2263499683,32cyclictest24666-21sh20:46:251
2149526810,35sleep30-21swapper/318:56:573
22642996745,13cyclictest0-21swapper/222:46:272
22642996745,12cyclictest0-21swapper/220:06:272
22642996741,14cyclictest0-21swapper/223:37:032
22642996710,22cyclictest26071-21seq20:49:002
22642996710,22cyclictest26071-21seq20:49:002
22634996739,17cyclictest0-21swapper/120:26:281
2263499673,41cyclictest13290-21dpkg20:11:281
2264799662,38cyclictest12793-21chrt00:09:053
22642996649,11cyclictest9723-21idleruntime-cro22:16:252
22642996649,11cyclictest9723-21idleruntime-cro22:16:252
22642996648,8cyclictest0-21swapper/223:06:492
22642996647,12cyclictest0-21swapper/222:37:002
22642996647,12cyclictest0-21swapper/222:37:002
22634996644,12cyclictest0-21swapper/123:37:031
2263499663,16cyclictest13924-21ssh23:16:501
22642996544,15cyclictest13993-21cpu00:11:402
22642996544,15cyclictest13993-21cpu00:11:402
22642996541,14cyclictest0-21swapper/221:36:292
22642996450,8cyclictest0-21swapper/200:06:532
22642996440,13cyclictest0-21swapper/219:06:292
2264299633,38cyclictest352-21runrttasks20:08:262
2264799627,35cyclictest0-21swapper/320:11:393
2264799627,35cyclictest0-21swapper/320:11:393
2264799582,44cyclictest2058-21taskset19:36:353
2264799553,42cyclictest6284-21taskset19:47:043
2264799553,38cyclictest6082-21taskset23:03:363
2264799553,38cyclictest6082-21taskset23:03:363
2264799543,37cyclictest17536-21chrt00:16:563
2264799542,38cyclictest25741-21chrt23:36:503
2264799532,39cyclictest9225-21taskset21:21:283
2264799532,39cyclictest31241-21taskset22:51:483
2264799532,36cyclictest22016-21chrt22:36:413
2264799532,36cyclictest22016-21chrt22:36:413
2264799523,38cyclictest17984-21chrt22:29:363
2264799523,36cyclictest20234-21chrt23:27:023
2264799523,35cyclictest21629-21taskset22:36:253
2264799523,34cyclictest5644-21taskset23:56:573
2264799523,31cyclictest32709-21chrt21:06:293
2264799522,37cyclictest25230-21chrt22:41:483
2264799522,36cyclictest25751-21chrt21:48:023
2264799522,35cyclictest6961-21chrt22:11:283
2264799513,39cyclictest22846-21taskset23:31:523
2264799513,37cyclictest2359-21taskset21:09:293
2264799513,37cyclictest10316-21chrt22:16:473
2264799513,36cyclictest19191-21chrt20:27:003
2264799513,33cyclictest12963-21chrt21:26:553
2264799512,35cyclictest30793-21chrt22:51:283
2264799512,33cyclictest28114-21taskset20:56:283
186712514,12sleep30-21swapper/321:36:493
2264799503,36cyclictest15697-21chrt23:20:253
2264799503,35cyclictest4116-21chrt19:41:493
2264799503,34cyclictest32645-21chrt19:31:253
2264799503,34cyclictest15628-21chrt20:17:083
2264799503,33cyclictest31079-21chrt19:26:273
2264799503,33cyclictest31079-21chrt19:26:273
2264799503,33cyclictest30294-21chrt23:44:323
2264799502,32cyclictest10531-21chrt23:11:423
2264799502,32cyclictest10531-21chrt23:11:423
2264799494,34cyclictest7384-21taskset22:11:493
2264799494,34cyclictest7384-21taskset22:11:493
2264799493,36cyclictest17376-21chrt20:21:543
2264799493,34cyclictest31844-21chrt21:57:393
2264799493,33cyclictest16665-21chrt00:16:273
2264799493,33cyclictest16665-21chrt00:16:273
2264799493,33cyclictest16006-21chrt21:31:583
2264799492,35cyclictest25421-21chrt00:31:193
2264799492,35cyclictest25421-21chrt00:31:193
2264799492,35cyclictest19837-21taskset00:21:343
2264799492,34cyclictest21520-21chrt20:36:273
283772483,6sleep30-21swapper/320:56:503
2264799484,34cyclictest7686-21chrt23:06:483
2264799483,33cyclictest2165-21chrt22:56:593
2264799483,31cyclictest9150-21chrt00:02:163
2264799483,31cyclictest9150-21chrt00:02:163
2264799483,30cyclictest6899-21chrt19:51:293
2264799482,37cyclictest23431-21chrt20:41:423
2264799482,33cyclictest25363-21chrt19:06:563
2264799482,33cyclictest25363-21chrt19:06:563
2264799473,35cyclictest10915-21chrt21:23:183
2264799473,33cyclictest16649-21taskset23:21:473
2264799473,31cyclictest26650-21chrt19:11:453
2264799473,31cyclictest26650-21chrt19:11:453
2264799472,33cyclictest14716-21chrt22:23:483
2264799472,33cyclictest14716-21chrt22:23:483
2264799472,33cyclictest11869-21chrt20:06:433
2264799472,32cyclictest4662-21chrt21:12:283
2264799472,32cyclictest28125-21chrt19:16:433
2264799472,32cyclictest28125-21chrt19:16:433
2264799472,31cyclictest10177-21chrt20:01:413
2264799463,34cyclictest23820-21taskset21:46:003
2264799463,34cyclictest22412-21taskset20:37:003
2264799463,30cyclictest1686-21chrt22:01:563
2264799463,30cyclictest1686-21chrt22:01:563
2264799452,33cyclictest9252-21chrt19:56:593
2264799452,30cyclictest628-21chrt19:31:453
2264799452,30cyclictest628-21chrt19:31:453
2264799452,30cyclictest22850-21chrt19:01:413
2264799442,29cyclictest25264-21chrt20:46:483
2264799442,29cyclictest25264-21chrt20:46:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional