You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-24 - 03:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Sun May 24, 2026 00:44:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
106089912957,64cyclictest14446-21acpi19:10:430
106089911340,8cyclictest0-21swapper/023:41:120
967221129,74sleep19699-21/usr/sbin/munin18:56:141
967221129,74sleep19699-21/usr/sbin/munin18:56:141
106089911135,13cyclictest0-21swapper/019:30:510
106089911135,13cyclictest0-21swapper/019:30:510
106089911036,12cyclictest0-21swapper/022:00:580
106089910932,10cyclictest0-21swapper/022:26:010
106089910917,30cyclictest23584-40gzip23:50:370
106089910842,15cyclictest15513-21apt-key21:50:360
106089910842,15cyclictest15513-21apt-key21:50:360
106089910837,14cyclictest0-21swapper/023:31:190
106089910837,14cyclictest0-21swapper/023:31:190
106089910837,12cyclictest0-21swapper/023:20:360
106089910835,11cyclictest0-21swapper/000:01:060
106089910738,8cyclictest0-21swapper/022:35:570
10608991072,31cyclictest2562-21grep20:11:080
10627991063,83cyclictest574-21apt-get00:05:482
10608991063,51cyclictest14799-21apt-get23:35:570
10608991063,51cyclictest14799-21apt-get23:35:570
10608991063,42cyclictest18294-21sendmail_mailqu19:21:050
10608991063,41cyclictest352-21runrttasks21:59:030
106089910633,12cyclictest0-21swapper/020:20:590
106089910614,29cyclictest13159-21idleruntime00:25:570
10627991054,60cyclictest3676-21apt-get21:30:442
10627991054,60cyclictest3676-21apt-get21:30:442
10608991054,42cyclictest24802-21kernelversion19:41:560
10608991054,34cyclictest12896-21idleruntime-cro20:45:320
10608991054,34cyclictest12896-21idleruntime-cro20:45:320
10608991053,41cyclictest352-21runrttasks00:08:390
106089910528,21cyclictest8508-21sort22:31:090
106089910528,21cyclictest8508-21sort22:31:090
106089910436,10cyclictest0-21swapper/023:56:090
10608991033,35cyclictest10952-21cut19:00:530
10608991033,35cyclictest10952-21cut19:00:530
106089910330,12cyclictest0-21swapper/022:11:020
106089910325,25cyclictest9462-21latency_hist21:40:350
10608991025,34cyclictest856-21fschecks_count21:25:540
10608991024,36cyclictest16544-21fschecks_count21:51:520
10608991024,34cyclictest31668-21find22:16:060
10608991024,34cyclictest31668-21find22:16:060
106089910239,9cyclictest0-21swapper/023:50:550
106089910239,8cyclictest0-21swapper/021:10:570
10608991023,37cyclictest18395-21tune2fs21:00:560
10608991014,39cyclictest12484-21latency_hist00:25:360
10608991014,36cyclictest6663-21/usr/sbin/munin20:26:090
10608991014,36cyclictest25740-21/usr/sbin/munin23:00:430
10608991014,34cyclictest9641-21/usr/sbin/munin21:41:010
106089910136,10cyclictest0-21swapper/019:06:300
10608991013,40cyclictest5541-21fschecks_count23:20:560
10608991013,38cyclictest30122-21cpu21:20:480
10608991013,37cyclictest352-21runrttasks23:29:530
10608991013,37cyclictest352-21runrttasks23:29:530
10608991004,33cyclictest31436-21latency_hist23:10:350
10608991003,43cyclictest1984-21ssh23:15:330
10098210013,63sleep09100-21/usr/sbin/munin18:56:210
10098210013,63sleep09100-21/usr/sbin/munin18:56:210
1060899994,35cyclictest9652-21latency_hist20:35:350
1060899994,35cyclictest9652-21latency_hist20:35:350
1060899994,34cyclictest3454-21acpi20:15:430
1060899993,74cyclictest58-21kcompactd000:11:000
1060899985,36cyclictest23980-21apt-key19:40:350
10608999826,18cyclictest17752-21diskmemload22:47:130
10608999817,28cyclictest28207-21fschecks_time19:51:560
10608999817,28cyclictest28207-21fschecks_time19:51:560
10608999817,28cyclictest28207-21fschecks_time19:51:560
10620999726,61cyclictest15181-21seq23:36:251
10620999726,61cyclictest15181-21seq23:36:251
1060899972,38cyclictest19621-21interrupts19:25:570
1060899972,38cyclictest19621-21interrupts19:25:570
10608999714,39cyclictest23787-21modprobe21:10:330
1060899964,37cyclictest1773-21fschecks_count22:20:550
1060899963,38cyclictest29965-21ntpq19:56:080
1060899963,38cyclictest29965-21ntpq19:56:080
1062799957,68cyclictest3869-21runrttasks00:10:502
310792940,7sleep20-21swapper/220:00:482
310792940,7sleep20-21swapper/220:00:482
1060899943,35cyclictest32029-21grep20:01:080
1060899943,35cyclictest32029-21grep20:01:080
1060899933,36cyclictest15602-21timerwakeupswit20:51:070
10627999272,13cyclictest23882-21latency_hist21:10:362
1062799913,7cyclictest0-21swapper/223:25:482
1062799913,7cyclictest0-21swapper/223:25:482
1062799913,35cyclictest6818-21apt-get00:15:442
1062799903,44cyclictest1552-21ssh21:26:092
10620998971,12cyclictest2-21kthreadd22:05:571
10620998966,16cyclictest277-21dbus-daemon22:20:341
10620998966,16cyclictest277-21dbus-daemon22:20:341
1062099893,77cyclictest32475-21apt-get00:05:481
1060899893,40cyclictest6368-21munin-run21:35:330
1060899893,40cyclictest6368-21munin-run21:35:330
1062799883,37cyclictest19058-21/usr/sbin/munin21:01:102
1062099883,73cyclictest3834-21apt-get00:10:501
1062099883,68cyclictest16039-21modprobe21:51:011
10627998768,13cyclictest2374-21systemd-journal22:35:342
10627998768,13cyclictest2374-21systemd-journal22:35:342
10627998729,50cyclictest0-21swapper/221:51:532
1062099879,49cyclictest31-21ksoftirqd/119:15:551
10620998762,16cyclictest14491-21timerwakeupswit22:41:091
10620998750,7cyclictest181rcu_preempt21:17:271
10620998750,7cyclictest181rcu_preempt21:17:271
10627998638,41cyclictest0-21swapper/223:10:322
1062799862,7cyclictest0-21swapper/222:00:482
1062799862,27cyclictest352-21runrttasks22:41:132
1062099868,59cyclictest25760-21apt-get23:01:161
1060899863,35cyclictest13128-21/usr/sbin/munin20:46:080
1062799853,7cyclictest0-21swapper/223:20:482
1062799853,6cyclictest0-21swapper/219:15:442
1062799853,38cyclictest31801-21sh23:10:462
1062799853,35cyclictest11563-21apt-get23:30:532
1062799853,35cyclictest11563-21apt-get23:30:532
1062799852,36cyclictest10529-21/usr/sbin/munin21:41:082
10627998444,32cyclictest0-21swapper/220:30:592
10627998444,32cyclictest0-21swapper/220:30:592
10627998441,36cyclictest0-21swapper/222:10:562
10627998441,36cyclictest0-21swapper/200:25:572
1062799843,7cyclictest0-21swapper/220:15:482
1062799843,34cyclictest27327-21runrttasks23:02:292
10620998415,35cyclictest31-21ksoftirqd/121:27:431
1062799833,37cyclictest25301-21cmp23:00:342
1062799833,37cyclictest25301-21cmp23:00:342
1062799833,35cyclictest23514-21unin-run23:50:342
1062799833,35cyclictest1620-21awk22:20:502
1062799832,30cyclictest12792-21cstates21:45:502
1062799832,30cyclictest12792-21cstates21:45:502
10627998240,35cyclictest0-21swapper/200:00:352
1062799823,39cyclictest27647-21sendmail22:10:332
1062799823,30cyclictest28847-21grep19:52:102
1062799823,30cyclictest28847-21grep19:52:102
1062799823,30cyclictest28847-21grep19:52:102
1062799823,27cyclictest17910-21cstates23:40:502
1062799822,6cyclictest0-21swapper/219:35:452
1062099827,34cyclictest31-21ksoftirqd/122:21:191
10620998221,28cyclictest0-21swapper/120:05:391
10620998216,25cyclictest301ktimers/123:42:331
1063299813,67cyclictest3913-21chrt00:10:573
1063299813,63cyclictest0-21swapper/300:05:483
10627998153,19cyclictest5104-21latency_hist23:20:352
10627998137,37cyclictest0-21swapper/219:50:342
10627998137,37cyclictest0-21swapper/219:50:342
10627998137,37cyclictest0-21swapper/219:41:562
1062799813,36cyclictest23940-21expr23:50:502
1062799813,35cyclictest32151-21ssh22:16:262
1062799813,35cyclictest32151-21ssh22:16:262
1062799813,31cyclictest11073-21/usr/sbin/munin22:36:032
10627998129,40cyclictest0-21swapper/219:00:542
10627998129,40cyclictest0-21swapper/219:00:542
1062799812,8cyclictest0-21swapper/221:10:472
10627998123,40cyclictest0-21swapper/200:01:062
1062799812,28cyclictest30085-21apt-get21:20:432
1062099818,44cyclictest31-21ksoftirqd/120:25:291
10620998157,18cyclictest14309-21latency_hist19:10:361
1062099811,46cyclictest31-21ksoftirqd/120:51:051
1062799803,33cyclictest20927-21http19:30:502
1062799803,33cyclictest20927-21http19:30:502
1062799803,33cyclictest19291-21taskset19:25:492
1062799803,33cyclictest19291-21taskset19:25:492
1062799803,33cyclictest12473-21/usr/sbin/munin20:41:092
1062799803,33cyclictest12473-21/usr/sbin/munin20:41:092
1062799803,30cyclictest14460-21seq19:10:442
1062799803,29cyclictest17490-21sh23:40:332
1062799803,29cyclictest17490-21sh23:40:332
1062799802,32cyclictest13312-21/usr/sbin/munin20:45:532
1062799802,31cyclictest7349-21date22:30:352
1062799802,31cyclictest29805-21/usr/sbin/munin19:56:042
1062799802,31cyclictest29805-21/usr/sbin/munin19:56:042
1062099804,51cyclictest27845-21apt-get19:50:531
1062099804,51cyclictest27845-21apt-get19:50:531
1062099804,51cyclictest27845-21apt-get19:50:531
10620998016,30cyclictest31-21ksoftirqd/119:41:021
1062799793,8cyclictest0-21swapper/222:45:462
1062799793,36cyclictest18964-21taskset21:55:482
1062799793,34cyclictest27860-21mailstats21:16:072
1062799793,34cyclictest27860-21mailstats21:16:072
1062799793,33cyclictest7625-21grep20:26:082
1062799793,30cyclictest18399-21cat19:21:072
1062799793,30cyclictest10879-21grep00:21:102
1062799793,29cyclictest2243-21/usr/sbin/munin20:10:592
1062799792,6cyclictest0-21swapper/220:55:462
1062799792,6cyclictest0-21swapper/220:55:462
10627997923,46cyclictest1764-21latency_hist20:10:362
1062799792,30cyclictest15111-21/usr/sbin/munin20:50:552
10620997954,19cyclictest13218-21idleruntime-cro22:40:321
10620997942,10cyclictest181rcu_preempt20:59:521
10620997942,10cyclictest181rcu_preempt20:59:521
10620997934,33cyclictest9641-21modprobe20:35:321
10620997934,33cyclictest9641-21modprobe20:35:321
1062799783,35cyclictest8896-21ssh21:39:182
1062799783,30cyclictest14163-21cron19:10:332
1062799783,29cyclictest19743-21/usr/sbin/munin22:50:532
1062799783,28cyclictest5835-21/usr/sbin/munin20:21:072
1062799782,29cyclictest10657-21latency_hist19:00:372
1062799782,29cyclictest10657-21latency_hist19:00:372
10620997862,10cyclictest22405-21kworker/u17:0-xprtiod23:51:061
10620997852,17cyclictest352-21runrttasks00:29:201
10620997850,8cyclictest181rcu_preempt22:32:201
10620997850,8cyclictest181rcu_preempt22:32:201
1062099782,43cyclictest31-21ksoftirqd/121:48:051
1062099782,43cyclictest31-21ksoftirqd/121:48:051
1062099782,39cyclictest31-21ksoftirqd/119:03:521
1062099782,39cyclictest31-21ksoftirqd/119:03:521
10620997741,9cyclictest181rcu_preempt21:23:271
1062099773,41cyclictest31-21ksoftirqd/122:52:421
1062099772,37cyclictest31-21ksoftirqd/100:17:421
10608997761,12cyclictest10598-21cyclictest21:20:200
10608997761,12cyclictest10598-21cyclictest21:20:200
10608997755,16cyclictest300-21rs:main0
1062799763,29cyclictest10900-21head20:36:092
1062799763,29cyclictest10900-21head20:36:092
10620997638,16cyclictest181rcu_preempt20:20:051
10620997616,11cyclictest301ktimers/123:47:131
10620997611,43cyclictest31-21ksoftirqd/122:02:111
10620997526,33cyclictest31-21ksoftirqd/123:31:161
10620997526,33cyclictest31-21ksoftirqd/123:31:161
1062099752,35cyclictest31-21ksoftirqd/119:58:561
1062099752,35cyclictest31-21ksoftirqd/119:58:561
10620997445,12cyclictest181rcu_preempt23:59:321
10620997441,11cyclictest181rcu_preempt23:10:381
10620997440,12cyclictest181rcu_preempt20:27:011
10620997422,20cyclictest0-21swapper/119:23:131
1060899744,55cyclictest352-21runrttasks00:19:020
926627310,37sleep30-21swapper/318:55:553
926627310,37sleep30-21swapper/318:55:553
10620997348,18cyclictest12341-21cron21:45:331
10620997347,8cyclictest181rcu_preempt21:58:561
1062099732,22cyclictest31-21ksoftirqd/123:28:241
1062099732,22cyclictest31-21ksoftirqd/123:28:241
1063299723,28cyclictest28798-21taskset23:05:523
1062099729,46cyclictest31-21ksoftirqd/120:42:211
1062099729,46cyclictest31-21ksoftirqd/120:42:211
10620997250,12cyclictest0-21swapper/123:06:171
10620997242,7cyclictest181rcu_preempt22:13:201
10620997240,15cyclictest181rcu_preempt22:28:521
10620997238,16cyclictest181rcu_preempt21:13:341
10620997238,12cyclictest181rcu_preempt21:38:071
10620997238,12cyclictest181rcu_preempt19:40:051
1062099723,52cyclictest23180-21/usr/sbin/munin22:56:041
1062099723,52cyclictest23180-21/usr/sbin/munin22:56:041
1062099723,45cyclictest10095-21fschecks_count20:35:541
1062099723,45cyclictest10095-21fschecks_count20:35:541
10620997234,10cyclictest181rcu_preempt19:28:191
10620997234,10cyclictest181rcu_preempt19:28:191
1062099722,44cyclictest31-21ksoftirqd/119:50:171
1062099722,44cyclictest31-21ksoftirqd/119:50:171
10608997256,11cyclictest13502-21kworker/u17:2+rpciod22:41:010
225642710,8sleep30-21swapper/319:35:503
10620997149,12cyclictest0-21swapper/100:01:111
10620997143,8cyclictest181rcu_preempt00:23:561
10620997138,12cyclictest181rcu_preempt20:13:431
1062099713,33cyclictest31-21ksoftirqd/123:23:021
10620997132,8cyclictest0-21swapper/121:00:471
1060899715,39cyclictest25658-21sendmail_mailst22:06:070
1063299703,28cyclictest19239-21chrt22:50:323
1063299703,26cyclictest15817-21taskset19:15:333
10620997037,12cyclictest181rcu_preempt21:32:121
10620997037,12cyclictest181rcu_preempt21:32:121
10620997035,14cyclictest181rcu_preempt22:49:571
1062099702,41cyclictest31-21ksoftirqd/120:47:111
10608997054,11cyclictest10598-21cyclictest19:16:020
1063299692,30cyclictest21190-21chrt22:52:393
1062099694,43cyclictest14629-21fschecks_count19:10:541
1062099691,23cyclictest31-21ksoftirqd/123:16:051
1060899693,28cyclictest25958-21/usr/sbin/munin19:46:100
1060899693,28cyclictest25958-21/usr/sbin/munin19:46:100
133772680,6sleep30-21swapper/322:40:363
1063299682,28cyclictest1502-21taskset21:26:083
10620996840,12cyclictest181rcu_preempt21:06:361
10608996852,10cyclictest23117-21sed22:56:010
10608996852,10cyclictest23117-21sed22:56:010
1063299673,29cyclictest18513-21chrt19:21:083
1063299673,25cyclictest22733-21chrt22:55:513
1063299673,25cyclictest22733-21chrt22:55:513
1063299673,24cyclictest5804-21taskset21:34:023
1063299673,24cyclictest5804-21taskset21:34:023
10608996746,11cyclictest0-21swapper/020:56:010
10608996746,11cyclictest0-21swapper/020:56:010
1063299663,27cyclictest9944-21taskset00:20:503
1063299663,26cyclictest23753-21chrt22:03:113
1063299663,24cyclictest12142-21chrt19:05:333
1063299663,24cyclictest12142-21chrt19:05:333
1063299662,27cyclictest4709-21chrt22:25:533
1063299662,27cyclictest15956-21taskset22:44:533
1062099669,22cyclictest31-21ksoftirqd/120:01:041
1062099669,22cyclictest31-21ksoftirqd/120:01:041
1060899666,49cyclictest10080-21runrttasks20:35:530
1060899666,49cyclictest10080-21runrttasks20:35:530
1060899663,41cyclictest955-21switchtime20:06:090
1063299653,29cyclictest13343-21chrt23:33:033
1063299653,29cyclictest13343-21chrt23:33:033
1063299653,27cyclictest21097-21taskset21:05:493
1063299653,25cyclictest32406-21chrt23:11:033
1063299653,25cyclictest20920-21chrt21:58:383
1063299653,23cyclictest20475-21chrt23:45:323
1063299652,23cyclictest3756-21chrt20:15:563
10632996511,44cyclictest0-21swapper/323:35:453
10632996511,44cyclictest0-21swapper/323:35:453
1062099652,35cyclictest31-21ksoftirqd/119:31:301
1062099652,35cyclictest31-21ksoftirqd/119:31:301
1063299643,27cyclictest563-21chrt22:18:513
1063299643,27cyclictest563-21chrt22:18:513
1063299643,24cyclictest7962-21chrt20:29:573
1063299643,24cyclictest31544-21taskset23:10:343
1063299643,23cyclictest28279-21taskset19:51:583
1063299643,23cyclictest28279-21taskset19:51:583
1063299643,23cyclictest28279-21taskset19:51:583
1063299643,23cyclictest27683-21chrt19:50:363
1063299643,23cyclictest27683-21chrt19:50:363
1063299642,26cyclictest28610-21taskset21:17:203
1063299642,26cyclictest28610-21taskset21:17:203
1063299642,26cyclictest14975-21chrt00:29:093
1063299642,24cyclictest1871-21chrt20:10:493
1063299642,23cyclictest28315-21chrt23:57:193
1063299642,23cyclictest21916-21taskset23:46:173
1063299642,23cyclictest10675-21taskset23:29:433
1063299642,23cyclictest10675-21taskset23:29:433
10632996413,41cyclictest0-21swapper/319:10:333
1063299633,22cyclictest25264-21chrt21:11:213
10632996311,43cyclictest0-21swapper/320:00:323
10632996311,43cyclictest0-21swapper/320:00:323
1063299623,27cyclictest32507-21chrt20:05:353
1063299623,27cyclictest32507-21chrt20:05:353
1063299623,25cyclictest26511-21chrt23:55:333
1063299623,22cyclictest9525-21chrt20:34:023
1063299623,22cyclictest9525-21chrt20:34:023
1063299623,21cyclictest30425-21chrt00:01:043
1063299622,25cyclictest9532-21chrt00:20:343
1063299613,24cyclictest10233-21chrt20:35:563
1063299613,24cyclictest10233-21chrt20:35:563
1063299613,21cyclictest27612-21chrt22:10:333
1063299612,24cyclictest15022-21chrt21:49:383
1063299612,24cyclictest15022-21chrt21:49:383
10632996110,42cyclictest0-21swapper/322:31:083
10632996110,42cyclictest0-21swapper/322:31:083
1063299603,26cyclictest11070-21taskset21:42:003
1063299603,24cyclictest17425-21chrt20:56:423
1063299603,24cyclictest17425-21chrt20:56:423
1063299603,23cyclictest16279-21chrt21:51:413
1063299603,22cyclictest25546-21taskset19:43:363
1063299602,25cyclictest16009-21chrt20:54:113
1063299602,22cyclictest7730-21chrt23:24:533
1063299593,23cyclictest3002-21chrt22:22:163
1063299593,20cyclictest3033-21chrt23:16:063
1063299592,23cyclictest7349-21chrt21:36:083
1063299592,23cyclictest12521-21chrt20:41:113
1063299592,23cyclictest12521-21chrt20:41:113
1063299592,22cyclictest19287-21chrt19:25:463
1063299592,22cyclictest19287-21chrt19:25:463
1063299592,21cyclictest6301-21taskset20:24:143
1063299583,23cyclictest19420-21chrt21:01:523
1063299583,19cyclictest22225-21chrt19:35:223
1063299583,19cyclictest22225-21chrt19:35:223
1063299582,24cyclictest1200-21taskset20:06:123
1063299582,21cyclictest29417-21chrt22:12:113
1063299582,19cyclictest13670-21chrt20:46:013
1063299572,22cyclictest30808-21chrt21:21:053
1063299572,22cyclictest17310-21taskset19:19:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional