You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 08:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Tue Apr 28, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20293991214,8cyclictest0-21swapper/000:07:350
20293991182,28cyclictest18839-21sh23:27:560
18826211612,77sleep018877-21sort18:57:380
18826211612,77sleep018877-21sort18:57:380
20293991153,73cyclictest0-21swapper/022:30:550
20293991153,73cyclictest0-21swapper/022:30:550
20293991153,29cyclictest6019-21/usr/sbin/munin23:07:420
20293991152,43cyclictest3206-21apt-get19:47:290
20293991143,29cyclictest5125-21runrttasks22:13:020
20293991142,23cyclictest27873-21ssh22:49:220
20293991133,27cyclictest1755-21ssh23:54:280
20293991123,26cyclictest8649-21ssh23:12:290
20293991123,26cyclictest8649-21ssh23:12:290
20293991123,23cyclictest8313-21uname22:17:420
20293991123,23cyclictest8313-21uname22:17:420
20293991122,26cyclictest6528-21runrttasks00:02:450
20293991122,24cyclictest6813-21ssh21:22:010
20293991113,24cyclictest18706-21seq22:34:590
20293991112,25cyclictest4248-21rm23:58:090
20293991112,24cyclictest2702-21taskset19:44:240
20293991112,24cyclictest23084-21rm00:30:400
20305991109,66cyclictest11828-21apt-get23:17:332
20293991103,79cyclictest15143-21rm00:17:320
20293991102,28cyclictest2722-21ssh23:02:310
20293991102,28cyclictest2722-21ssh23:02:310
20293991093,24cyclictest24810-21ssh22:44:030
20293991093,24cyclictest24810-21ssh22:44:030
20293991092,27cyclictest2884-21ssh21:14:200
20293991092,26cyclictest17369-21ssh23:27:110
20293991092,26cyclictest11609-21taskset21:28:440
20293991092,22cyclictest29337-21ssh21:58:080
20293991083,29cyclictest1672-21ssh23:00:160
20293991083,22cyclictest13559-21rm23:19:430
20293991082,24cyclictest22473-21ssh23:34:470
20293991074,23cyclictest26009-21fschecks_count20:57:390
20293991072,25cyclictest30815-21ssh23:48:200
20293991072,25cyclictest1074-21ssh22:06:320
20293991072,24cyclictest23286-21ssh21:47:580
20293991063,27cyclictest17279-21seq20:30:070
20293991063,27cyclictest17279-21seq20:30:070
20293991063,24cyclictest3272-21ssh22:09:170
20293991063,24cyclictest21491-21taskset19:03:200
20293991063,24cyclictest21491-21taskset19:03:200
20293991062,27cyclictest26342-21ssh21:53:070
20293991062,27cyclictest26342-21ssh21:53:070
20293991062,27cyclictest23671-21taskset20:49:120
20293991062,24cyclictest28915-21ssh21:03:470
20293991062,24cyclictest28915-21ssh21:03:470
20293991062,24cyclictest13177-21ssh22:26:160
20293991055,90cyclictest22021-21latency_hist21:47:210
20293991055,90cyclictest22021-21latency_hist21:47:210
20293991053,25cyclictest9683-21ssh21:26:420
20293991053,25cyclictest9683-21ssh21:26:420
20293991053,23cyclictest352-21runrttasks23:42:150
20293991052,24cyclictest22514-21ssh22:41:370
20293991052,24cyclictest22514-21ssh22:41:370
20293991052,23cyclictest13716-21taskset00:14:510
20293991043,23cyclictest25297-21taskset20:53:590
20293991042,25cyclictest28628-21sh23:45:300
20293991042,23cyclictest32734-21ssh21:10:280
20293991042,23cyclictest30225-21taskset19:28:210
20293991042,23cyclictest30225-21taskset19:28:210
20293991033,25cyclictest352-21runrttasks19:13:520
20293991032,27cyclictest12467-21taskset20:15:140
20293991032,23cyclictest24021-21taskset19:11:320
20293991024,26cyclictest17645-21ssh21:38:520
20293991023,24cyclictest1234-21seq19:41:140
20293991022,26cyclictest31976-21seq19:34:550
20293991022,24cyclictest27253-21runrttasks19:21:230
20293991013,24cyclictest20613-21ssh00:27:020
20293991013,21cyclictest22008-21seq20:43:270
20293991002,27cyclictest19676-21snmpd20:35:560
20293991002,23cyclictest30569-21ssh22:53:310
20297999847,45cyclictest2374-21systemd-journal20:58:221
2029399982,25cyclictest28913-21seq19:27:090
2029399982,22cyclictest12883-21seq20:17:230
20297999621,66cyclictest0-21swapper/122:37:421
20297999621,66cyclictest0-21swapper/122:37:421
2029399963,7cyclictest0-21swapper/019:57:340
2029399963,7cyclictest0-21swapper/019:57:340
2029399962,22cyclictest8040-21seq20:02:290
2029399962,22cyclictest14827-21ssh21:34:210
20297999549,38cyclictest0-21swapper/121:52:461
20297999549,38cyclictest0-21swapper/121:52:461
20297999445,41cyclictest27699-21cstates19:22:341
20297999428,36cyclictest0-21swapper/123:33:031
2029399943,23cyclictest20387-21runrttasks20:38:170
20297999350,36cyclictest19526-21latency_hist22:37:201
20297999346,41cyclictest3993-21timerandwakeup19:47:531
2029799933,76cyclictest0-21swapper/100:02:311
20293999334,53cyclictest352-21runrttasks20:07:520
2029399933,31cyclictest14497-21/usr/sbin/munin20:22:370
2029399933,31cyclictest14497-21/usr/sbin/munin20:22:370
20297999251,33cyclictest0-21swapper/100:18:541
2029799923,57cyclictest277-21dbus-daemon20:12:181
20297999061,19cyclictest22513-21latency_hist20:47:211
20297999042,40cyclictest0-21swapper/119:12:451
2029399902,24cyclictest6396-21date19:57:200
2029399902,24cyclictest6396-21date19:57:200
20297998848,34cyclictest27183-21diskmemload22:10:591
20297998839,41cyclictest0-21swapper/121:32:411
2029799882,65cyclictest0-21swapper/119:57:341
2029799882,65cyclictest0-21swapper/119:57:341
20305998734,13cyclictest181rcu_preempt00:27:532
20297998749,32cyclictest31325-21sh21:08:451
20297998746,34cyclictest30694-21sh22:53:541
20297998742,36cyclictest0-21swapper/119:47:211
2020528715,44sleep21-21systemd19:02:212
2020528715,44sleep21-21systemd19:02:212
20305998667,13cyclictest2790-21unin-run23:57:182
20305998667,12cyclictest26445-21df_inode22:47:362
20297998563,14cyclictest17935-21kworker/u17:2-rpciod23:29:401
20297998538,39cyclictest0-21swapper/123:37:431
2029799853,55cyclictest32236-40ogrotate23:52:171
20305998417,31cyclictest31455-21fschecks_time22:02:392
2029799846,57cyclictest20138-21kworker/u19:0+ext4-rsv-conversion23:17:501
20297998456,20cyclictest23539-21sh21:48:411
2029799844,67cyclictest0-21swapper/121:27:291
20305998357,18cyclictest1-21systemd21:46:222
20305998357,18cyclictest1-21systemd21:46:222
20297998364,13cyclictest2374-21systemd-journal21:24:221
20297998364,13cyclictest2374-21systemd-journal21:24:221
20297998360,18cyclictest14980-21kworker/u17:0+events_unbound23:47:161
20297998355,20cyclictest30807-21sh22:02:171
20297998325,32cyclictest0-21swapper/123:09:491
20305998245,8cyclictest181rcu_preempt20:12:322
2030599823,66cyclictest22593-21munin-node19:07:292
2030599822,66cyclictest15124-21apt-get00:17:522
2029799825,61cyclictest4986-21modprobe22:12:381
20297998242,35cyclictest14132-21kworker/u17:1+events_unbound20:53:171
2029799823,52cyclictest17628-21latency_hist23:27:201
20297998152,20cyclictest27183-21diskmemload23:02:511
20297998152,20cyclictest27183-21diskmemload23:02:511
2029799813,59cyclictest32212-21apt-get22:57:291
20297998123,49cyclictest0-21swapper/119:57:201
20297998123,49cyclictest0-21swapper/119:57:201
20305998062,12cyclictest22755-21apt-get22:42:172
20305998062,12cyclictest22755-21apt-get22:42:172
2030599803,65cyclictest4927-21apt-get22:12:412
20305998030,9cyclictest181rcu_preempt23:46:232
20297998064,11cyclictest20280-21cyclictest00:12:451
20297998061,14cyclictest20280-21cyclictest22:26:521
2029799803,65cyclictest0-21swapper/100:07:301
20297998035,39cyclictest0-21swapper/119:07:211
20297998035,39cyclictest0-21swapper/119:07:211
2029799802,67cyclictest0-21swapper/120:32:301
2029799802,67cyclictest0-21swapper/120:12:291
20305997957,17cyclictest544-21cron23:22:202
20305997940,13cyclictest181rcu_preempt22:36:202
20297997937,35cyclictest14613-21cstates20:22:351
20297997937,35cyclictest14613-21cstates20:22:351
20297997937,34cyclictest0-21swapper/122:02:381
20297997932,37cyclictest0-21swapper/122:51:521
20305997857,16cyclictest58-21kcompactd020:58:052
20305997848,20cyclictest4501-21apt-key22:12:202
20297997856,12cyclictest0-21swapper/122:27:321
20297997856,12cyclictest0-21swapper/122:27:321
20297997852,17cyclictest24101-21latency_hist20:52:201
20297997840,31cyclictest0-21swapper/100:27:541
20297997839,32cyclictest0-21swapper/121:02:421
20297997839,32cyclictest0-21swapper/121:02:421
2029799782,69cyclictest0-21swapper/120:02:311
20305997745,12cyclictest181rcu_preempt19:53:382
20305997745,12cyclictest181rcu_preempt19:53:382
20297997753,15cyclictest298-21in:imuxsock21:42:181
20297997742,28cyclictest0-21swapper/100:22:381
20297997727,42cyclictest0-21swapper/121:17:421
20305997658,12cyclictest14630-21systemd-run00:17:182
20305997656,14cyclictest2374-21systemd-journal22:57:222
20305997649,21cyclictest24537-21sh19:12:372
20305997636,11cyclictest181rcu_preempt21:14:582
20305997610,57cyclictest23417-21ssh22:42:472
20305997610,57cyclictest23417-21ssh22:42:472
20297997654,16cyclictest1-21systemd22:21:041
20297997654,16cyclictest1-21systemd22:21:041
20297997637,32cyclictest0-21swapper/123:52:501
20297997636,33cyclictest0-21swapper/121:12:511
2029799763,60cyclictest0-21swapper/120:27:321
2029799763,60cyclictest0-21swapper/120:27:321
20305997555,14cyclictest30723-21apt-get21:08:242
20305997546,12cyclictest181rcu_preempt23:30:552
20305997542,12cyclictest181rcu_preempt21:37:532
2030599752,22cyclictest26930-21unixbench_singl19:17:542
20305997510,48cyclictest20840-21sh23:32:332
20297997535,32cyclictest0-21swapper/120:17:471
2029799752,39cyclictest0-21swapper/122:44:051
2029799752,39cyclictest0-21swapper/122:44:051
20305997448,16cyclictest12555-21ssh22:24:312
20305997424,19cyclictest8396-21latency_hist23:12:192
2029799743,61cyclictest23257-21/usr/sbin/munin19:08:411
2031199733,35cyclictest0-21swapper/300:17:493
20305997353,12cyclictest11354-21sh21:28:022
20305997351,14cyclictest0-21swapper/200:22:412
2030599734,42cyclictest29689-21cstates23:47:332
2029799733,56cyclictest0-21swapper/119:32:291
2029799732,34cyclictest0-21swapper/119:30:451
2029799732,34cyclictest0-21swapper/119:30:451
20305997251,14cyclictest22257-21kworker/u17:3-rpciod23:57:512
20305997237,9cyclictest181rcu_preempt20:37:552
20305997237,12cyclictest181rcu_preempt21:25:082
20305997237,12cyclictest181rcu_preempt21:25:082
20305997229,13cyclictest181rcu_preempt00:02:502
2029799723,64cyclictest0-21swapper/119:17:291
2029799723,63cyclictest0-21swapper/123:57:301
2029799722,64cyclictest0-21swapper/119:37:311
2029799722,38cyclictest9862-21ssh23:13:161
2029799722,38cyclictest9862-21ssh23:13:161
20305997148,14cyclictest26267-21ssh23:42:122
20305997141,10cyclictest181rcu_preempt19:47:312
2030599713,45cyclictest2981-21runrttasks23:02:402
2030599713,45cyclictest2981-21runrttasks23:02:402
20305997126,32cyclictest31111-21fschecks_time19:32:412
2030599712,45cyclictest25186-21grep20:52:522
2029799712,40cyclictest20575-21ssh21:43:441
2029799712,40cyclictest20575-21ssh21:43:441
134142710,4sleep3451rcuc/321:32:383
20305997048,13cyclictest0-21swapper/219:47:202
2030599703,36cyclictest23414-21timerwakeupswit20:47:532
20305997033,12cyclictest181rcu_preempt20:33:192
20305997021,12cyclictest0-21swapper/220:27:402
20305997021,12cyclictest0-21swapper/220:27:402
20297997028,35cyclictest0-21swapper/120:37:361
200502705,17sleep30-21swapper/319:01:123
200502705,17sleep30-21swapper/319:01:123
20305996933,20cyclictest13828-21/usr/sbin/munin22:27:472
20305996933,20cyclictest13828-21/usr/sbin/munin22:27:472
20305996932,13cyclictest181rcu_preempt00:07:422
20305996930,12cyclictest181rcu_preempt21:49:032
20305996927,14cyclictest181rcu_preempt22:01:062
2030599692,43cyclictest39-21ksoftirqd/222:57:142
2030599692,29cyclictest25508-21rm21:52:412
2030599692,29cyclictest25508-21rm21:52:412
20305996849,10cyclictest181rcu_preempt22:19:042
20305996849,10cyclictest181rcu_preempt22:19:042
20305996847,11cyclictest0-21swapper/220:17:502
20305996828,8cyclictest181rcu_preempt21:20:252
2030599682,29cyclictest28425-21cat19:22:522
20305996813,30cyclictest8443-21grep20:02:442
20305996811,37cyclictest20960-21/usr/sbin/munin20:42:372
201902688,37sleep120204-21munin-run19:02:201
201902688,37sleep120204-21munin-run19:02:201
20305996744,16cyclictest1331-21munin-plugin-st19:42:192
2030599673,42cyclictest11589-21latency_hist23:17:202
2030599673,42cyclictest11589-21latency_hist23:17:202
20305996734,12cyclictest181rcu_preempt21:36:122
2030599669,11cyclictest15440-21unixbench_multi20:22:542
2030599669,11cyclictest15440-21unixbench_multi20:22:542
2030599664,34cyclictest28234-21awk21:02:522
2030599664,34cyclictest28234-21awk21:02:522
2030599664,25cyclictest11054-21modprobe20:12:182
2030599663,47cyclictest6635-21fschecks_count19:57:402
2030599663,47cyclictest6635-21fschecks_count19:57:402
20305996624,30cyclictest29862-21cat19:27:512
20305996624,30cyclictest29862-21cat19:27:512
2031199643,41cyclictest0-21swapper/322:42:443
2031199643,41cyclictest0-21swapper/322:42:443
20305996442,12cyclictest0-21swapper/219:07:212
20305996442,12cyclictest0-21swapper/219:07:212
2031199639,31cyclictest0-21swapper/322:42:363
2031199639,31cyclictest0-21swapper/322:42:363
20311996314,30cyclictest0-21swapper/323:17:343
20311996210,34cyclictest0-21swapper/322:12:413
2031199613,41cyclictest20012-21taskset22:37:213
2031199583,31cyclictest0-21swapper/319:07:173
2031199583,31cyclictest0-21swapper/319:07:173
2031199573,43cyclictest8540-21chrt00:07:023
2031199568,41cyclictest0-21swapper/319:08:263
2031199553,42cyclictest14985-21taskset23:22:363
2031199553,37cyclictest19981-21chrt20:37:513
2031199543,39cyclictest12905-21taskset21:32:173
2031199543,38cyclictest3774-21chrt19:47:503
2031199543,37cyclictest7216-21taskset23:09:153
2031199532,38cyclictest23734-21taskset00:32:193
2031199532,34cyclictest18015-21taskset00:22:223
2031199523,37cyclictest2064-21chrt21:12:523
2031199523,35cyclictest21075-21taskset20:42:333
2031199523,33cyclictest4725-21chrt22:12:213
2031199523,32cyclictest24168-21taskset20:52:213
2031199522,36cyclictest30759-21chrt21:07:363
2031199522,35cyclictest30305-21taskset21:07:203
2031199522,35cyclictest30305-21taskset21:07:203
2031199513,36cyclictest29030-21taskset23:46:403
2031199513,35cyclictest26742-21chrt22:47:433
2031199513,34cyclictest29013-21chrt19:27:203
2031199513,34cyclictest24483-21chrt20:52:373
2031199513,34cyclictest15058-21chrt20:22:493
2031199513,34cyclictest15058-21chrt20:22:493
2031199513,32cyclictest3637-21chrt23:57:463
2031199512,33cyclictest32595-21chrt19:37:373
20311995036,10cyclictest451rcuc/323:02:553
20311995036,10cyclictest451rcuc/323:02:553
2031199503,34cyclictest32663-21chrt22:04:063
2031199503,34cyclictest22789-21taskset23:35:423
2031199503,33cyclictest16498-21chrt22:32:153
2031199503,33cyclictest16498-21chrt22:32:153
2031199503,22cyclictest0-21swapper/323:52:213
2031199502,37cyclictest6494-21taskset21:21:063
2031199494,31cyclictest1501-21taskset23:53:453
2031199493,38cyclictest23502-21taskset21:48:343
2031199493,37cyclictest19118-21chrt23:28:453
2031199493,33cyclictest13485-21chrt00:14:113
2031199493,32cyclictest9062-21chrt22:18:053
2031199493,32cyclictest9062-21chrt22:18:053
2031199493,32cyclictest20627-21chrt21:43:523
2031199493,32cyclictest20627-21chrt21:43:523
2031199492,33cyclictest10613-21taskset00:09:293
2031199483,33cyclictest17842-21chrt21:39:263
2031199483,32cyclictest27094-21chrt19:19:223
2031199483,32cyclictest18023-21taskset20:32:413
2031199483,32cyclictest10911-21chrt23:16:163
2031199483,32cyclictest10911-21chrt23:16:163
2031199483,31cyclictest27104-21taskset21:55:153
2031199483,31cyclictest27104-21taskset21:55:153
2031199483,30cyclictest30913-21taskset19:32:353
2031199482,34cyclictest16775-21taskset20:27:513
2031199482,34cyclictest16775-21taskset20:27:513
2031199482,33cyclictest7870-21taskset20:02:193
2031199482,33cyclictest7870-21taskset20:02:193
2031199482,33cyclictest24733-21chrt23:37:573
2031199482,32cyclictest8480-21taskset20:02:443
2031199473,36cyclictest28366-21taskset21:57:383
2031199473,35cyclictest1774-21taskset19:42:393
2031199473,31cyclictest10429-21chrt20:07:543
2031199472,32cyclictest14349-21chrt20:22:203
2031199472,30cyclictest7196-21taskset21:22:243
2031199472,30cyclictest7196-21taskset21:22:243
2031199463,29cyclictest10998-21chrt22:22:343
2031199462,35cyclictest4962-21chrt19:52:353
2031199462,35cyclictest4962-21chrt19:52:353
2031199462,35cyclictest31441-21chrt22:56:023
2031199462,30cyclictest26074-21chrt20:57:403
2031199453,31cyclictest11859-21chrt20:12:483
2031199452,30cyclictest25783-21chrt19:17:203
20311994432,8cyclictest451rcuc/323:00:573
2031199442,31cyclictest29494-21chrt19:27:413
2031199442,31cyclictest29494-21chrt19:27:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional