You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-25 - 06:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7.osadl.org (updated Wed Mar 25, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7621991042,31cyclictest8407-21ssh21:47:001
7616991022,71cyclictest15-21ksoftirqd/022:09:510
76219910078,15cyclictest641-21latency_hist22:29:361
7616991003,69cyclictest11697-21apt-get19:14:490
7628999660,30cyclictest14738-21diskmemload00:08:212
65842967,62sleep16587-21/usr/sbin/munin19:00:081
7616999532,55cyclictest9311-21latency_hist21:49:370
7621999374,11cyclictest5169-21timerwakeupswit23:30:111
7621999374,11cyclictest5169-21timerwakeupswit23:30:111
762899924,32cyclictest15035-21/usr/sbin/munin21:05:022
762899923,32cyclictest8033-21tune2fs00:29:552
762899923,26cyclictest1402-21df23:24:522
7628999143,40cyclictest0-21swapper/219:40:092
762899913,36cyclictest13924-21wc19:20:062
762899913,35cyclictest9755-21cat20:49:362
762199913,72cyclictest9628-21apt-get20:50:021
7628999049,8cyclictest181rcu_preempt23:39:202
762899904,30cyclictest15573-21/usr/sbin/munin22:00:092
762899903,33cyclictest757-21sed22:29:372
7621999042,41cyclictest18937-21timerandwakeup19:35:081
312442900,3sleep20-21swapper/220:14:552
762899894,36cyclictest352-21runrttasks22:49:582
762899894,35cyclictest11904-21sh21:53:322
762899894,33cyclictest18838-40mandb23:54:372
762899893,34cyclictest27383-21/usr/sbin/munin22:20:072
762899893,33cyclictest13231-21latency_hist19:19:372
762899892,39cyclictest3838-21ls21:39:512
762899892,39cyclictest3838-21ls21:39:512
762899892,35cyclictest7441-21ls20:40:072
762899892,33cyclictest8617-21timerwakeupswit19:05:102
7621998966,15cyclictest10122-21fschecks_time22:44:581
7621998966,15cyclictest10122-21fschecks_time22:44:581
7628998875,8cyclictest0-21swapper/200:10:002
7628998875,8cyclictest0-21swapper/200:10:002
7628998875,8cyclictest0-21swapper/200:10:002
762899884,67cyclictest27245-21apt-get21:24:492
762899884,67cyclictest27245-21apt-get21:24:492
762899883,36cyclictest18473-21latency_hist22:59:352
762899883,36cyclictest18473-21latency_hist22:59:352
762899883,34cyclictest6785-21cstates21:44:512
7628998833,49cyclictest0-21swapper/223:14:592
7628998833,49cyclictest0-21swapper/223:14:592
7628998832,49cyclictest0-21swapper/223:09:502
7628998827,9cyclictest0-21swapper/200:24:422
7628998827,9cyclictest0-21swapper/200:24:422
7621998842,39cyclictest27886-21latency22:20:011
762199883,36cyclictest15355-21/usr/sbin/munin19:25:001
762199883,31cyclictest27409-21ssh21:25:121
762199883,31cyclictest27409-21ssh21:25:121
635728826,36sleep30-21swapper/319:00:023
762899873,37cyclictest13116-21/usr/sbin/munin23:44:562
762899873,37cyclictest13116-21/usr/sbin/munin23:44:562
762899873,35cyclictest22861-21latency_hist19:49:362
762899873,33cyclictest24107-21awk22:14:332
762899873,24cyclictest18018-21/usr/sbin/munin21:10:072
762899872,34cyclictest1159-21uname21:34:592
7621998742,35cyclictest0-21swapper/121:39:371
7621998739,41cyclictest24397-21latency_hist19:54:361
762199872,61cyclictest0-21swapper/120:44:331
762199872,34cyclictest1491-21latency_hist00:19:361
7616998764,17cyclictest7595-21latency_hist00:29:360
7616998764,17cyclictest7595-21latency_hist00:29:360
761699874,45cyclictest15-21ksoftirqd/023:26:570
761699873,51cyclictest15-21ksoftirqd/020:18:250
762899864,34cyclictest27034-21latency_hist21:24:382
762899864,30cyclictest21514-21apt-key22:09:382
762899863,36cyclictest10425-21fschecks_time20:50:502
762899863,32cyclictest16806-21idleruntime19:29:582
7628998629,8cyclictest0-21swapper/222:35:082
762899862,34cyclictest28040-21tune2fs20:04:582
7628998617,23cyclictest23122-21awk00:00:102
7628998617,23cyclictest23122-21awk00:00:102
762199865,8cyclictest0-21swapper/120:04:521
7621998641,37cyclictest0-21swapper/121:50:001
762199863,34cyclictest1739-21/usr/sbin/munin23:25:001
762199863,32cyclictest30311-21ssh00:12:461
762199863,32cyclictest30311-21ssh00:12:461
762199863,32cyclictest30311-21ssh00:12:461
762199862,28cyclictest12907-21expr22:49:501
761699869,69cyclictest16723-21kworker/u18:0+ext4-rsv-conversion20:49:520
762899854,33cyclictest9784-21/usr/sbin/munin22:45:002
762899854,33cyclictest9784-21/usr/sbin/munin22:45:002
762899854,33cyclictest352-21runrttasks00:21:182
762899854,31cyclictest23871-21grep19:50:082
762899853,36cyclictest4079-21mailstats20:30:072
762899853,34cyclictest18133-21/usr/sbin/munin19:35:072
762899853,32cyclictest30555-21/usr/sbin/munin21:29:572
7628998525,47cyclictest0-21swapper/220:29:082
7628998513,14cyclictest0-21swapper/219:24:502
762199854,6cyclictest0-21swapper/122:59:481
7621998541,36cyclictest0-21swapper/100:29:551
762199852,8cyclictest0-21swapper/120:59:511
762199852,8cyclictest0-21swapper/120:59:511
762199852,33cyclictest12242-21/usr/sbin/munin20:55:011
7616998542,37cyclictest30952-21latency_hist00:14:440
7616998542,37cyclictest30952-21latency_hist00:14:440
7616998542,37cyclictest30952-21latency_hist00:14:440
761699853,42cyclictest15-21ksoftirqd/022:41:450
761699852,43cyclictest15-21ksoftirqd/023:21:000
63002859,40sleep20-21swapper/219:00:012
762899845,36cyclictest19102-21/usr/sbin/munin23:55:032
762899844,34cyclictest23733-21taskset21:19:332
762899844,33cyclictest19003-21fschecks_count22:59:562
762899843,37cyclictest10057-21fschecks_time19:09:582
762899843,37cyclictest10057-21fschecks_time19:09:582
762899843,33cyclictest352-21runrttasks23:39:572
762899843,33cyclictest22721-21unixbench_multi23:05:102
762899843,33cyclictest14710-21cron21:04:332
762899843,33cyclictest14710-21cron21:04:332
762899842,35cyclictest5584-21ntp_kernel_pll_20:35:012
762899842,33cyclictest3776-21taskset22:34:372
7628998420,57cyclictest0-21swapper/223:30:002
7628998420,57cyclictest0-21swapper/223:30:002
7621998443,33cyclictest0-21swapper/120:49:351
7621998441,35cyclictest0-21swapper/121:19:561
762199843,36cyclictest27091-21/usr/sbin/munin20:00:091
762199843,32cyclictest21271-21ssh21:14:541
762199843,31cyclictest20397-21rm22:07:191
7616998433,27cyclictest20609-21runrttasks22:07:550
762899834,35cyclictest26791-21grep20:00:052
762899834,33cyclictest30694-21/usr/sbin/munin23:20:072
762899833,35cyclictest12871-21fschecks_count21:54:552
7628998330,48cyclictest7609-21cyclictest22:19:372
7628998330,48cyclictest7609-21cyclictest22:19:372
762899832,33cyclictest11748-21/usr/sbin/munin20:54:512
7621998342,33cyclictest0-21swapper/100:24:541
7621998342,33cyclictest0-21swapper/100:24:541
7616998367,10cyclictest1501-21ssh22:30:040
761699835,46cyclictest15-21ksoftirqd/020:43:280
761699835,40cyclictest15-21ksoftirqd/021:34:270
7616998342,32cyclictest0-21swapper/022:24:560
7628998228,9cyclictest0-21swapper/222:40:022
7628998225,6cyclictest0-21swapper/200:16:092
762199825,36cyclictest32461-21ssh21:33:281
762199823,7cyclictest0-21swapper/123:19:471
762199823,6cyclictest0-21swapper/121:09:471
762199823,34cyclictest5887-21/usr/sbin/munin20:35:081
762199823,34cyclictest25981-21unin-run19:59:351
762199823,33cyclictest4489-21/usr/sbin/munin21:40:091
762199823,33cyclictest4489-21/usr/sbin/munin21:40:091
762199823,32cyclictest13644-21/usr/sbin/munin19:19:571
762199822,37cyclictest9930-21latency_hist23:39:351
762199822,36cyclictest22097-21/usr/sbin/munin23:59:571
762199822,36cyclictest22097-21/usr/sbin/munin23:59:571
762199822,35cyclictest11172-21apt-get19:14:491
761699828,52cyclictest18901-21sed23:54:360
761699823,56cyclictest15-21ksoftirqd/020:39:020
761699822,49cyclictest15-21ksoftirqd/022:19:330
761699822,49cyclictest15-21ksoftirqd/022:19:330
613028213,46sleep06132-21apt-get18:59:470
762899813,36cyclictest32665-21find20:19:442
762899813,32cyclictest29784-21sed20:09:562
762899813,31cyclictest25085-21ntp_kernel_err19:55:022
7621998160,13cyclictest21976-21ssh23:04:551
7621998135,39cyclictest0-21swapper/121:59:521
762199813,36cyclictest17038-21ntp_offset19:30:031
762199813,35cyclictest28863-21ssh23:15:241
762199813,35cyclictest28863-21ssh23:15:241
762199812,37cyclictest1554-21/usr/sbin/munin22:30:061
762199812,35cyclictest2677-21/usr/sbin/munin00:20:101
762199812,32cyclictest7261-21ssh22:39:581
761699816,41cyclictest12461-21modprobe22:49:330
761699816,41cyclictest12461-21modprobe22:49:330
761699814,53cyclictest15-21ksoftirqd/021:22:410
761699814,40cyclictest352-21runrttasks23:07:060
7621998061,12cyclictest277-21dbus-daemon20:19:361
7621998040,33cyclictest200-21systemd-journal00:07:551
762199803,9cyclictest0-21swapper/119:44:461
762199803,7cyclictest0-21swapper/120:09:441
762199803,34cyclictest15070-21sh21:04:471
762199803,33cyclictest2000-21/usr/sbin/munin20:24:561
762199802,35cyclictest666-21/usr/sbin/munin20:20:001
762199802,35cyclictest25010-21ssh23:09:551
762199802,34cyclictest9986-21/usr/sbin/munin19:09:571
762199802,34cyclictest9986-21/usr/sbin/munin19:09:571
762199802,34cyclictest20011-21sed23:55:071
762199802,32cyclictest18789-21latency_hist23:54:371
761699804,30cyclictest15-21ksoftirqd/000:07:100
762199794,8cyclictest0-21swapper/122:14:481
762199794,8cyclictest0-21swapper/122:14:481
762199793,7cyclictest0-21swapper/122:34:461
762199793,34cyclictest14187-21grep23:45:091
762199793,34cyclictest14187-21grep23:45:091
762199793,33cyclictest3645-21fschecks_count20:29:561
762199793,33cyclictest20686-21/usr/sbin/munin19:40:091
762199793,31cyclictest13059-21latency_hist23:44:371
762199793,30cyclictest22881-21/usr/sbin/munin22:11:301
761699794,35cyclictest15-21ksoftirqd/020:11:440
7616997929,43cyclictest0-21swapper/021:10:090
762199783,34cyclictest7867-21/usr/sbin/munin19:04:511
762199782,32cyclictest15317-21ssh21:59:351
761699785,37cyclictest15-21ksoftirqd/023:59:130
761699782,44cyclictest15-21ksoftirqd/020:27:240
762199772,32cyclictest15973-21fschecks_count22:54:551
762199772,32cyclictest15973-21fschecks_count22:54:551
7616997749,21cyclictest24314-21idleruntime-cro19:54:340
761699773,42cyclictest15-21ksoftirqd/019:42:370
7616997657,14cyclictest7609-21cyclictest21:15:110
7616997653,15cyclictest9122-21idleruntime-cro19:09:330
761699765,29cyclictest15-21ksoftirqd/023:36:040
7616997652,19cyclictest352-21runrttasks23:29:390
7616997652,19cyclictest352-21runrttasks23:29:390
7616997651,19cyclictest544-21cron19:14:340
7616997651,19cyclictest544-21cron19:14:340
7616997651,18cyclictest8012-21fschecks_count00:29:550
7616997649,19cyclictest28867-21ssh23:15:240
7616997649,19cyclictest28867-21ssh23:15:240
761699763,51cyclictest15-21ksoftirqd/021:02:350
761699763,51cyclictest15-21ksoftirqd/021:02:350
761699763,44cyclictest15-21ksoftirqd/020:59:270
761699762,48cyclictest15-21ksoftirqd/020:21:290
761699754,40cyclictest16165-21ntpq22:00:030
761699753,39cyclictest15-21ksoftirqd/023:13:220
761699753,35cyclictest15-21ksoftirqd/023:04:160
761699753,29cyclictest15-21ksoftirqd/022:58:110
761699753,29cyclictest15-21ksoftirqd/022:58:110
7616997447,18cyclictest27977-21ntp_kernel_pll_22:20:030
761699744,38cyclictest15-21ksoftirqd/019:34:030
761699744,29cyclictest15-21ksoftirqd/023:49:100
761699744,29cyclictest15-21ksoftirqd/023:49:100
7616997355,11cyclictest14738-21diskmemload21:25:270
7616997355,11cyclictest14738-21diskmemload21:25:270
761699735,38cyclictest15-21ksoftirqd/019:46:020
7616997347,17cyclictest24289-21ssh00:03:140
7616997347,17cyclictest24289-21ssh00:03:140
761699734,33cyclictest1640-21timerandwakeup21:35:100
7616997340,24cyclictest4964-21grep22:35:110
761699733,31cyclictest15-21ksoftirqd/000:19:500
761699732,36cyclictest15-21ksoftirqd/019:58:160
761699723,42cyclictest31809-21fschecks_count00:15:410
7616997151,14cyclictest10078-21latency21:49:590
761699714,35cyclictest15054-21apt-get21:04:450
761699713,36cyclictest15-21ksoftirqd/020:05:100
761699704,45cyclictest15-21ksoftirqd/020:29:390
761699704,35cyclictest15-21ksoftirqd/021:56:530
7616997043,19cyclictest14766-21latency_hist19:24:360
761699703,39cyclictest15-21ksoftirqd/019:39:010
761699693,20cyclictest15-21ksoftirqd/023:43:020
761699688,9cyclictest0-21swapper/020:49:330
761699682,55cyclictest27554-21apt20:04:330
761699682,35cyclictest3834-21cstates21:39:520
761699682,35cyclictest3834-21cstates21:39:520
7616996748,13cyclictest14762-21ssh22:52:450
763499652,40cyclictest27332-21taskset21:24:593
763499652,40cyclictest27332-21taskset21:24:593
761699655,33cyclictest15-21ksoftirqd/019:27:020
763499613,36cyclictest0-21swapper/320:50:023
763499599,32cyclictest0-21swapper/322:09:483
763499594,37cyclictest26168-21taskset19:59:373
763499587,26cyclictest0-21swapper/319:14:493
262222573,40sleep30-21swapper/319:59:503
763499542,40cyclictest8171-21taskset20:44:363
763499542,39cyclictest10778-21taskset19:10:133
763499542,39cyclictest10778-21taskset19:10:133
763499534,35cyclictest4932-21chrt00:24:493
763499534,35cyclictest4932-21chrt00:24:493
763499533,37cyclictest13079-21taskset23:44:373
763499533,37cyclictest13079-21taskset23:44:373
763499533,36cyclictest1405-21chrt22:30:013
763499532,40cyclictest21077-21chrt21:14:463
763499532,40cyclictest17949-21chrt23:52:153
763499523,36cyclictest6758-21chrt23:34:203
763499523,36cyclictest6758-21chrt23:34:203
763499523,36cyclictest16408-21chrt19:29:363
763499523,36cyclictest1374-21taskset00:19:143
763499523,32cyclictest12685-21chrt21:54:493
763499522,38cyclictest11989-21chrt22:48:173
763499522,38cyclictest11989-21chrt22:48:173
7634995136,8cyclictest451rcuc/319:31:183
763499513,37cyclictest8093-21chrt00:29:563
763499513,35cyclictest509-21chrt21:34:363
763499513,35cyclictest16787-21chrt22:00:293
763499512,36cyclictest24776-21chrt23:09:483
763499512,36cyclictest11823-21chrt23:41:453
763499503,35cyclictest8074-21chrt23:35:093
763499503,35cyclictest754-21taskset22:29:373
763499503,35cyclictest19433-21chrt22:05:113
763499503,33cyclictest31327-21chrt20:14:563
763499502,38cyclictest24551-21chrt00:03:583
763499502,38cyclictest24551-21chrt00:03:583
763499502,36cyclictest28151-21chrt23:14:593
763499502,36cyclictest28151-21chrt23:14:593
763499502,36cyclictest1527-21taskset21:35:083
763499502,35cyclictest8422-21chrt21:47:033
763499502,35cyclictest18256-21chrt19:34:513
763499493,33cyclictest858-21chrt23:24:273
763499493,33cyclictest10050-21taskset21:49:593
763499493,32cyclictest8657-21chrt19:05:103
763499492,35cyclictest2509-21chrt00:20:073
763499492,33cyclictest5684-21chrt21:42:423
763499492,33cyclictest5684-21chrt21:42:423
763499492,33cyclictest4323-21chrt22:35:003
763499492,33cyclictest19375-21chrt21:10:493
763499484,35cyclictest27036-21chrt22:19:273
763499484,35cyclictest27036-21chrt22:19:273
763499483,35cyclictest5947-21chrt20:35:093
763499483,34cyclictest32733-21chrt20:19:503
763499483,32cyclictest24366-21chrt21:19:563
763499483,31cyclictest15244-21taskset21:04:533
763499482,36cyclictest31113-21chrt00:14:403
763499482,36cyclictest31113-21chrt00:14:403
763499482,36cyclictest31113-21chrt00:14:403
763499482,34cyclictest28145-21chrt22:20:083
763499482,30cyclictest28218-21chrt00:09:373
763499473,31cyclictest15405-21chrt22:54:333
763499472,35cyclictest14181-21taskset19:20:093
763499472,33cyclictest28386-21chrt20:05:083
763499472,32cyclictest4885-21chrt20:34:363
763499472,32cyclictest3773-21chrt23:29:173
763499464,29cyclictest22922-21chrt19:49:373
763499463,32cyclictest13888-21chrt21:00:033
763499463,32cyclictest13888-21chrt21:00:033
763499462,32cyclictest2202-21taskset20:25:003
763499452,30cyclictest23795-21chrt23:07:583
763499443,32cyclictest18196-21chrt22:58:583
763499443,32cyclictest18196-21chrt22:58:583
763499442,33cyclictest12052-21chrt20:54:573
763499442,32cyclictest19236-21chrt23:00:003
763499442,29cyclictest20729-21chrt19:40:093
763499434,27cyclictest9195-21chrt22:43:433
763499432,33cyclictest20867-21chrt23:56:573
763499402,26cyclictest23634-21chrt19:50:063
763499392,29cyclictest29871-21taskset20:09:583
763499392,26cyclictest8838-21chrt20:45:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional