You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 07:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7.osadl.org (updated Thu Dec 25, 2025 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29641991215,24cyclictest29643-21latency_hist19:09:530
29641991215,24cyclictest29643-21latency_hist19:09:530
29641991185,37cyclictest17156-21/usr/sbin/munin22:25:110
29641991095,29cyclictest5503-21/usr/sbin/munin23:55:160
29647991084,77cyclictest4484-21apt-get19:30:031
296419910855,46cyclictest32325-21fschecks_time23:45:100
296419910855,46cyclictest32325-21fschecks_time23:45:100
29641991073,37cyclictest346-21runrttasks23:27:100
29641991073,28cyclictest19575-21runrttasks21:35:040
702421050,5sleep10-21swapper/121:14:391
29641991044,30cyclictest10723-21latency_hist19:49:510
296419910442,54cyclictest0-21swapper/022:04:510
296419910442,54cyclictest0-21swapper/022:04:510
296419910431,67cyclictest0-21swapper/023:40:000
29641991024,31cyclictest2379-21apt19:24:480
29641991023,37cyclictest11209-21fschecks_time19:50:100
29641991017,26cyclictest28762-21tune2fs21:50:110
29641991017,26cyclictest28762-21tune2fs21:50:110
29641991013,39cyclictest15271-21sort00:10:240
29641991012,24cyclictest22280-21/usr/sbin/munin20:25:120
296419910110,12cyclictest0-21swapper/022:14:510
296419910044,48cyclictest0-21swapper/021:17:070
29641991004,33cyclictest8666-21cstates00:00:090
29641991004,27cyclictest29012-21cat23:39:520
29641991003,91cyclictest0-21swapper/020:40:020
29641991003,46cyclictest786-21sendmail-msp19:19:480
29641991003,33cyclictest26840-21modprobe20:39:480
29654999947,47cyclictest0-21swapper/223:10:242
29647999972,18cyclictest13836-21latency_hist23:14:511
2964199994,35cyclictest20606-21fschecks_count00:20:080
2964199994,33cyclictest32718-21unixbench_singl22:50:260
2964199994,33cyclictest32718-21unixbench_singl22:50:260
29641999937,55cyclictest19871-21timerwakeupswit20:15:250
2964199992,32cyclictest346-21runrttasks21:56:270
2964199992,30cyclictest23735-21irqstats00:25:130
2964199992,30cyclictest23735-21irqstats00:25:130
29641999854,38cyclictest8500-21rm23:05:120
2964199984,37cyclictest3210-21unixbench_singl22:55:260
2964199984,31cyclictest346-21runrttasks22:18:440
2964199984,12cyclictest0-21swapper/021:29:530
2964199984,12cyclictest0-21swapper/021:29:530
2964199983,37cyclictest23690-21sed22:35:240
2964199983,32cyclictest0-21swapper/023:30:040
2964199983,30cyclictest25411-21latency_hist21:44:520
2964199983,29cyclictest13231-21latency_hist21:24:510
2964199983,22cyclictest29392-21cstates00:35:040
2947529815,55sleep30-21swapper/319:09:483
2947529815,55sleep30-21swapper/319:09:483
2964199974,38cyclictest5229-21/usr/sbin/munin23:54:480
2964199974,36cyclictest20390-21ntpq22:30:200
2964199974,36cyclictest20390-21ntpq22:30:200
29641999736,52cyclictest17761-21if_eth000:15:110
29641999736,52cyclictest17761-21if_eth000:15:110
2964199973,38cyclictest7710-21latency_hist22:09:500
29641999724,8cyclictest0-21swapper/021:10:050
2964199972,31cyclictest25729-21apt-get22:40:000
29641999720,12cyclictest0-21swapper/020:30:240
29641999720,12cyclictest0-21swapper/020:30:240
2964199964,34cyclictest32034-21cpuspeed20:55:050
2964199964,32cyclictest2864-21cstates21:05:050
2964199964,32cyclictest2864-21cstates21:05:050
2964199963,35cyclictest9237-21latency_hist19:44:510
29654999546,43cyclictest4126-21diskmemload23:40:262
2964199954,35cyclictest17179-21latency_hist20:09:500
2964199954,35cyclictest17179-21latency_hist20:09:500
2964199954,33cyclictest14756-21timerandwakeup22:20:240
2964199953,35cyclictest346-21runrttasks00:04:570
2964199953,35cyclictest346-21runrttasks00:04:570
2964199953,34cyclictest28662-21/usr/sbin/munin22:44:590
2964199953,34cyclictest2703-21ls19:25:040
2964199953,33cyclictest5184-21mailstats19:30:580
2964199953,29cyclictest13419-21cat19:55:250
2964199953,26cyclictest14213-21/usr/sbin/munin20:00:240
2964199953,26cyclictest14213-21/usr/sbin/munin20:00:240
29641999515,21cyclictest28555-21sh00:33:470
29647999467,18cyclictest5999-21threads23:00:241
29647999467,18cyclictest5999-21threads23:00:241
29647999439,43cyclictest31836-21latency_hist23:44:511
2964199944,33cyclictest16990-21/usr/sbin/munin23:20:190
2964199944,32cyclictest346-21runrttasks21:26:080
2964199944,32cyclictest346-21runrttasks21:26:080
2964199944,31cyclictest30548-21fschecks_count20:50:120
2964199944,31cyclictest22022-21modprobe20:24:480
2964199944,30cyclictest6122-21/usr/sbin/munin19:35:060
2964199944,29cyclictest25908-21kernelversion21:45:120
2964199943,34cyclictest5117-21/usr/sbin/munin23:00:140
2964199943,34cyclictest5117-21/usr/sbin/munin23:00:140
2964199943,33cyclictest11493-21seq23:10:140
2964199943,32cyclictest28979-21fschecks_time20:45:110
2964199943,29cyclictest31271-21latency_hist19:14:510
2964199943,26cyclictest15044-21grep23:15:270
29641999410,26cyclictest18270-21cat20:10:240
29641999410,26cyclictest18270-21cat20:10:240
29641999312,28cyclictest2579-21latency_hist21:04:510
281732938,60sleep228251-21cstates19:05:072
281732938,60sleep228251-21cstates19:05:072
228502930,7sleep10-21swapper/100:24:221
2965499913,65cyclictest19639-21ssh21:35:152
2964799892,63cyclictest19719-21cpuspeed21:35:171
29654998838,44cyclictest12621-21sh00:05:332
29654998838,44cyclictest12621-21sh00:05:332
283362888,56sleep128412-21fschecks_time19:05:141
283362888,56sleep128412-21fschecks_time19:05:141
29654998717,32cyclictest346-21runrttasks23:58:402
29654998646,34cyclictest346-21runrttasks21:10:132
2965499863,73cyclictest4955-21rm22:05:002
2965499853,43cyclictest29989-21ssh00:35:202
29647998558,19cyclictest1-21systemd20:14:071
29647998558,19cyclictest1-21systemd20:14:071
2965499843,73cyclictest0-21swapper/219:09:592
29654998339,34cyclictest0-21swapper/221:05:072
29654998339,34cyclictest0-21swapper/221:05:072
29647998361,14cyclictest201-21systemd-journal22:09:071
29654998245,31cyclictest15539-21runrttasks00:10:272
29654998242,33cyclictest0-21swapper/220:45:112
29654998238,37cyclictest22610-21latency_hist23:29:512
2965999813,45cyclictest0-21swapper/321:35:153
29654998143,31cyclictest23835-21diskmemload22:35:262
29654998143,31cyclictest0-21swapper/200:15:042
29654998143,31cyclictest0-21swapper/200:15:042
29654998038,35cyclictest0-21swapper/221:49:482
141962790,22sleep1301ktimers/122:20:111
29654997842,31cyclictest29634-21cyclictest22:42:262
29654997840,31cyclictest0-21swapper/220:30:102
29654997840,31cyclictest0-21swapper/220:30:102
29654997837,33cyclictest0-21swapper/219:54:502
29654997836,35cyclictest201-21systemd-journal22:14:062
29654997826,44cyclictest14280-21cut23:15:112
2965499782,47cyclictest346-21runrttasks23:01:022
2965499782,47cyclictest346-21runrttasks23:01:022
29647997863,10cyclictest201-21systemd-journal23:32:071
29647997856,16cyclictest23602-21fschecks_time00:25:101
29647997856,16cyclictest23602-21fschecks_time00:25:101
2965499773,64cyclictest0-21swapper/219:40:022
29654997642,28cyclictest4126-21diskmemload00:20:082
29654997638,32cyclictest29634-21cyclictest19:59:512
29654997637,31cyclictest0-21swapper/220:29:512
2965499763,44cyclictest346-21runrttasks22:59:082
29654997633,37cyclictest0-21swapper/200:00:192
29647997652,13cyclictest0-21swapper/122:30:131
29647997652,13cyclictest0-21swapper/122:30:131
29647997627,25cyclictest4126-21diskmemload21:28:081
29647997627,25cyclictest4126-21diskmemload21:28:081
2965499753,44cyclictest23279-21ssh00:24:542
2965499753,44cyclictest23279-21ssh00:24:542
29654997533,34cyclictest0-21swapper/221:20:272
29654997532,34cyclictest0-21swapper/220:19:482
29654997531,36cyclictest0-21swapper/222:03:022
29654997531,36cyclictest0-21swapper/222:03:022
29654997531,36cyclictest0-21swapper/221:50:292
29654997531,36cyclictest0-21swapper/221:50:292
29654997433,33cyclictest0-21swapper/223:06:342
29654997429,37cyclictest0-21swapper/200:30:002
29654997428,35cyclictest0-21swapper/223:36:552
2965499742,43cyclictest346-21runrttasks22:28:122
29647997455,12cyclictest29634-21cyclictest22:00:001
29647997455,12cyclictest29634-21cyclictest22:00:001
29647997448,19cyclictest19303-21latency_hist21:34:511
29647997448,19cyclictest19303-21latency_hist21:34:511
29647997445,20cyclictest9090-21ntp_kernel_err00:00:201
29654997341,26cyclictest19143-21kworker/u17:2-rpciod22:50:232
29654997341,26cyclictest19143-21kworker/u17:2-rpciod22:50:232
29654997335,31cyclictest0-21swapper/222:30:052
29654997335,31cyclictest0-21swapper/222:30:052
29654997334,32cyclictest0-21swapper/222:47:212
29654997333,33cyclictest0-21swapper/221:40:522
29654997329,38cyclictest0-21swapper/221:19:482
29654997329,34cyclictest19361-21apt-config21:34:522
29654997329,34cyclictest19361-21apt-config21:34:522
29647997356,10cyclictest26190-21cpuspeed23:35:051
29647997355,13cyclictest17711-21processes23:20:201
29647997345,17cyclictest0-21swapper/120:00:271
29647997345,17cyclictest0-21swapper/120:00:271
29654997253,13cyclictest10037-21mailstats19:45:212
29654997250,15cyclictest29634-21cyclictest22:20:112
2965499723,64cyclictest0-21swapper/220:50:032
29654997233,31cyclictest0-21swapper/220:40:222
29654997229,35cyclictest0-21swapper/222:19:492
29654997228,38cyclictest0-21swapper/219:35:152
2965499722,43cyclictest4496-21modprobe19:30:052
29647997251,12cyclictest0-21swapper/121:00:231
29647997250,16cyclictest14663-21open_files23:15:221
29647997238,11cyclictest181rcu_preempt00:14:581
29647997238,11cyclictest181rcu_preempt00:14:581
29654997131,32cyclictest0-21swapper/219:20:242
29654997128,37cyclictest0-21swapper/223:20:132
2965499712,58cyclictest22032-21ls20:24:482
2965499712,53cyclictest1005-21ssh23:46:172
2965499712,53cyclictest1005-21ssh23:46:172
29647997153,13cyclictest32696-21switchtime20:55:221
29647997151,14cyclictest29634-21cyclictest19:17:081
29647997148,15cyclictest29634-21cyclictest23:09:471
2964799712,24cyclictest346-21runrttasks23:47:551
2964799712,24cyclictest346-21runrttasks23:47:551
29654997033,30cyclictest0-21swapper/219:26:452
29654997032,31cyclictest0-21swapper/223:30:472
29654997023,40cyclictest201-21systemd-journal21:27:432
29654997023,40cyclictest201-21systemd-journal21:27:432
2964799705,44cyclictest32406-21awk22:50:231
2964799705,44cyclictest32406-21awk22:50:231
29647997044,16cyclictest201-21systemd-journal20:30:071
29647997044,16cyclictest201-21systemd-journal20:30:071
29654996947,15cyclictest1-21systemd21:02:072
29647996945,13cyclictest0-21swapper/122:47:591
29647996924,33cyclictest382-21rs:main1
29654996846,15cyclictest5355-21ssh23:54:512
2965499683,59cyclictest0-21swapper/220:35:042
2965499682,61cyclictest0-21swapper/221:54:592
29647996851,11cyclictest3093-21ssh23:50:161
2965499679,51cyclictest15906-21ls20:05:072
2965499679,51cyclictest15906-21ls20:05:072
2965499673,56cyclictest17779-21/usr/sbin/munin20:10:132
2965499673,56cyclictest17779-21/usr/sbin/munin20:10:132
29654996728,32cyclictest0-21swapper/219:19:512
2965499672,55cyclictest15212-21/usr/sbin/munin20:00:282
2965499672,55cyclictest15212-21/usr/sbin/munin20:00:282
29647996749,12cyclictest0-21swapper/100:05:101
29647996749,12cyclictest0-21swapper/100:05:101
29647996746,11cyclictest0-21swapper/121:45:091
2964799673,47cyclictest7502-21systemctl19:39:491
2964799673,44cyclictest31325-21head20:50:271
2964799673,44cyclictest27106-21seq22:41:131
2964799672,46cyclictest2601-21latency_hist19:24:521
29647996713,22cyclictest19143-21kworker/u17:2+xprtiod22:56:211
29654996624,37cyclictest0-21swapper/220:55:212
29647996644,14cyclictest8166-21latency_hist23:59:511
2964799664,38cyclictest32052-21latency_hist00:39:511
29647996640,18cyclictest22798-21fschecks_count21:40:111
29647996550,8cyclictest0-21swapper/119:45:141
29647996547,9cyclictest0-21swapper/121:15:261
29647996545,14cyclictest15507-21unixbench_singl00:10:261
2964799653,45cyclictest2935-21idleruntime19:25:101
250872650,6sleep30-21swapper/323:32:503
29647996447,9cyclictest0-21swapper/119:14:481
29647996348,9cyclictest13942-21munin-run19:59:491
29647996348,9cyclictest0-21swapper/121:20:161
29647996342,15cyclictest675-21sh21:57:081
29647996331,21cyclictest12496-21latency_hist19:54:521
29647996322,29cyclictest16880-21sshd20:06:071
29647996322,29cyclictest16880-21sshd20:06:071
29647996321,31cyclictest7923-21sshd19:40:081
29647996317,12cyclictest0-21swapper/122:13:231
29647996245,11cyclictest1375-21kworker/u17:1-ext4-rsv-conversion22:29:511
2964799624,43cyclictest20409-21munin-run20:19:481
29647996238,13cyclictest0-21swapper/100:30:101
2964799623,38cyclictest22786-21latency_hist23:29:511
29647996142,11cyclictest0-21swapper/121:09:511
29647996142,11cyclictest0-21swapper/121:09:511
29647996140,13cyclictest0-21swapper/120:25:171
29647996139,12cyclictest0-21swapper/122:15:101
2964799613,39cyclictest22966-21fschecks_count22:35:101
29647996042,12cyclictest28874-21kernelversion21:50:141
29647996042,12cyclictest28874-21kernelversion21:50:141
29647995934,15cyclictest0-21swapper/120:40:211
29659995811,30cyclictest0-21swapper/319:30:033
29647995843,8cyclictest0-21swapper/120:35:081
29647995833,15cyclictest0-21swapper/120:45:111
2965999542,38cyclictest29781-21chrt22:45:253
29659995337,12cyclictest451rcuc/321:05:183
29659995337,12cyclictest451rcuc/321:05:183
2965999532,38cyclictest5338-21taskset23:54:503
2965999524,35cyclictest9505-21chrt19:45:063
29659995240,8cyclictest451rcuc/322:27:463
2965999523,36cyclictest7602-21chrt22:09:483
2965999523,36cyclictest17989-21taskset21:31:593
2965999523,36cyclictest17989-21taskset21:31:593
2965999523,33cyclictest16678-21chrt22:24:513
2965999522,38cyclictest14850-21chrtset21:26:283
2965999522,38cyclictest14850-21chrtset21:26:283
2965999522,37cyclictest24550-21chrt00:25:463
2965999522,37cyclictest24550-21chrt00:25:463
29659995139,8cyclictest451rcuc/323:35:403
29659995138,9cyclictest451rcuc/323:43:253
2965999513,38cyclictest22941-21taskset20:25:223
2965999513,35cyclictest30487-21taskset20:50:083
2965999513,34cyclictest25601-21chrt20:35:063
2965999512,34cyclictest1856-21chrt23:48:433
2965999512,34cyclictest1856-21chrt23:48:433
2965999508,31cyclictest2561-21chrt22:55:143
2965999504,37cyclictest32668-21chrt21:55:503
2965999504,34cyclictest20148-21chrt20:16:483
29659995037,9cyclictest451rcuc/323:28:273
2965999503,35cyclictest6059-21chrt23:00:243
2965999503,35cyclictest6059-21chrt23:00:243
2965999503,33cyclictest9241-21chrt23:05:313
2965999503,32cyclictest11498-21chrt22:15:193
2965999503,31cyclictest27233-21chrt20:40:043
2965999493,37cyclictest6880-21chrt19:35:223
2965999493,35cyclictest8235-21taskset19:40:163
2965999493,35cyclictest13839-21chrt23:14:513
2965999493,34cyclictest23764-21chrt21:40:473
2965999493,31cyclictest5074-21chrt21:10:203
2965999492,32cyclictest9144-21chrt21:17:133
2965999492,30cyclictest30261-21chrt20:49:513
2965999487,33cyclictest25032-21chrt20:32:143
2965999487,33cyclictest25032-21chrt20:32:143
29659994835,9cyclictest451rcuc/322:00:093
29659994835,9cyclictest451rcuc/322:00:093
2965999483,35cyclictest19880-21chrt23:24:523
2965999483,33cyclictest30522-21chrt00:35:553
2965999483,33cyclictest13067-21chrt00:06:503
2965999483,33cyclictest13067-21chrt00:06:503
2965999483,32cyclictest3491-21chrt19:25:223
2965999483,31cyclictest15784-21taskset23:17:183
2965999483,29cyclictest21170-21chrt22:31:153
2965999483,29cyclictest21170-21chrt22:31:153
2965999483,25cyclictest20331-21chrt00:19:543
2965999474,29cyclictest7966-21chrt23:59:163
2965999473,34cyclictest12897-21chrt21:24:223
2965999473,32cyclictest954-21chrt19:19:513
2965999473,32cyclictest9149-21chrt22:10:493
2965999472,33cyclictest27810-21chrt21:48:533
2965999472,32cyclictest30178-21chrt21:52:093
2965999472,32cyclictest30178-21chrt21:52:093
2965999472,32cyclictest27119-21chrt00:30:243
2965999464,33cyclictest32098-21taskset20:55:053
2965999463,34cyclictest17790-21chrt00:15:133
2965999463,34cyclictest17790-21chrt00:15:133
2965999462,32cyclictest16321-21taskset20:05:183
2965999462,32cyclictest16321-21taskset20:05:183
2965999453,33cyclictest26351-21chrt22:40:193
2965999452,33cyclictest10359-21taskset00:02:323
2965999443,28cyclictest14085-21chrt19:59:523
2965999433,26cyclictest23303-21taskset22:35:163
2965999432,32cyclictest31955-21chrt22:50:113
2965999432,32cyclictest31955-21chrt22:50:113
2965999423,31cyclictest16748-21chrt00:13:553
29659994228,10cyclictest451rcuc/320:12:003
29659994228,10cyclictest451rcuc/320:12:003
29659994024,8cyclictest451rcuc/321:00:123
2965999393,24cyclictest14204-21taskset19:59:563
2965999393,24cyclictest14204-21taskset19:59:563
2965999392,28cyclictest22065-21taskset20:24:493
2965999392,27cyclictest11597-21chrt19:50:203
2965999383,26cyclictest2144-21taskset19:21:583
2965999382,27cyclictest30876-21chrt19:11:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional