You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-18 - 07:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Mon May 18, 2026 00:45:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
170611899974,4cyclictest131rcu_preempt00:40:023
1706105999524,5cyclictest309-21in:imuxsock22:40:010
1706105999226,12cyclictest1808007-21ssh00:30:120
1706105998818,65cyclictest16850irq/134-800000021:32:210
1706105998420,12cyclictest1712835-21ssh21:10:400
1706105998415,63cyclictest16850irq/134-800000021:59:160
1706105998330,35cyclictest1766618-21ssh23:03:590
1706105998330,35cyclictest1766618-21ssh23:03:590
1706105998317,61cyclictest16850irq/134-800000023:08:380
1706105998221,15cyclictest1725658-21ssh21:38:020
1706105998215,62cyclictest16850irq/134-800000021:17:570
1706105998020,13cyclictest1744615-21ssh22:17:490
1706105997923,15cyclictest1739881-21ssh22:07:540
170610599783,3cyclictest120-21kswapd023:15:010
170610599783,3cyclictest120-21kswapd023:15:010
1706105997827,15cyclictest1796469-21ssh00:06:070
1706105997825,18cyclictest1743192-21ssh22:15:010
1706105997825,12cyclictest1787551-21ssh23:47:300
1706105997728,14cyclictest1756414-21ssh22:42:290
1706105997727,15cyclictest1736361-21ssh22:00:050
170610599771,37cyclictest1784021-21latency_hist23:40:010
1706105997629,41cyclictest275-21dbus-daemon21:05:010
170610599761,10cyclictest1752887-21latency_hist22:35:020
1706105997541,30cyclictest16850irq/134-800000021:25:010
1706105997427,15cyclictest1759275-21ssh22:48:330
1706105997419,12cyclictest1777888-21ssh23:27:180
1706105997416,13cyclictest1772621-21ssh23:16:090
1706105997415,49cyclictest16850irq/134-800000021:48:080
1706105997412,51cyclictest16850irq/134-800000000:02:490
1706105997322,7cyclictest1727566-21ssh21:41:500
1706105997318,6cyclictest16850irq/134-800000022:51:230
1706105997318,6cyclictest16850irq/134-800000022:51:230
1706105997224,14cyclictest1800785-21ssh00:15:030
1706105997220,11cyclictest1720070-21ssh21:25:560
170610599721,39cyclictest1733663-21ssh21:55:010
1706105997128,8cyclictest1764637-21ssh22:59:530
170610599711,66cyclictest0-21swapper/020:40:010
170610599711,66cyclictest0-21swapper/020:40:000
1706105997018,47cyclictest16850irq/134-800000022:23:280
1706105997016,48cyclictest16850irq/134-800000023:59:140
1706105996930,35cyclictest16950irq/224-800000000:22:540
1706105996827,15cyclictest1805983-21ssh00:25:570
1706105996815,48cyclictest16850irq/134-800000000:15:020
1706105996735,29cyclictest1708921-21munin-plugin-st20:04:590
1706118996622,39cyclictest37-21ksoftirqd/322:26:213
1706118996622,39cyclictest37-21ksoftirqd/322:26:213
1706108996654,9cyclictest1706049-21latency_hist19:10:011
1706105996617,14cyclictest1748916-21ssh22:26:410
1706105996617,14cyclictest1748916-21ssh22:26:410
1706118996541,19cyclictest1-21systemd22:15:013
170611899652,6cyclictest1774282-21munin-run23:20:013
1706105996529,33cyclictest1789189-21rm23:50:510
1706105996529,33cyclictest1789189-21rm23:50:510
1706118996421,5cyclictest1746721-21sh22:21:573
1706105996424,7cyclictest12-21ksoftirqd/020:30:000
1706118996324,35cyclictest1734713-21sh21:57:013
170610899635,36cyclictest23-21ksoftirqd/122:00:021
1706105996333,27cyclictest16850irq/134-800000023:25:010
1706105996333,27cyclictest16850irq/134-800000023:25:010
1706118996216,41cyclictest1717226-21cat21:20:023
1706105996228,29cyclictest16950irq/224-800000023:35:010
1706105996228,29cyclictest16950irq/224-800000023:35:010
170611899612,53cyclictest1745598-21cron22:20:003
1706118996120,14cyclictest275-21dbus-daemon23:45:013
1706118996120,14cyclictest275-21dbus-daemon23:45:013
170610599612,23cyclictest1712373-21latency_hist21:10:020
1706118996026,30cyclictest1793428-21latency_hist00:00:023
1706118996026,29cyclictest1757045-21sh22:44:013
1706118996021,35cyclictest1710318-21cat20:30:013
1706118996018,4cyclictest1712283-21diskmemload23:10:013
1706105996023,11cyclictest12-21ksoftirqd/019:50:010
1706118995925,31cyclictest1762610-21sh22:55:223
1706118995925,30cyclictest37-21ksoftirqd/322:02:193
1706118995923,32cyclictest1712283-21diskmemload21:10:473
1706118995914,10cyclictest1771671-1kworker/u9:023:38:363
170611899591,15cyclictest1708626-21cron20:00:013
1706113995951,5cyclictest1769784-21kworker/2:023:20:452
1706113995951,5cyclictest1769784-21kworker/2:023:20:452
1706113995948,6cyclictest1709976-21kworker/2:021:40:022
170610599592,4cyclictest1810737-21sshd00:35:560
1706118995822,32cyclictest1755303-21latency_hist22:40:023
1706118995822,30cyclictest1776850-21date23:25:023
1706118995822,30cyclictest1776850-21date23:25:023
170611899582,16cyclictest1731284-21ssh21:50:003
1706108995834,11cyclictest1774261-21cron23:20:001
1706105995829,25cyclictest16950irq/224-800000023:40:070
1706105995829,25cyclictest16950irq/224-800000023:40:070
1706118995723,30cyclictest1739068-21sh22:06:033
1706118995720,9cyclictest1709674-21cron20:20:023
1706118995720,9cyclictest1709674-21cron20:20:023
1706118995715,6cyclictest0-21swapper/321:42:463
1706105995737,15cyclictest3467602-21systemd-journal20:35:010
1706118995617,8cyclictest1647312-1kworker/u9:121:22:013
1706118995614,23cyclictest1781594-21ssh23:35:023
1706118995614,23cyclictest1781594-21ssh23:35:023
1706113995648,5cyclictest1709976-21kworker/2:021:27:342
1706113995647,5cyclictest1709976-21kworker/2:021:35:022
1706113995646,6cyclictest1769784-21kworker/2:023:42:202
1706113995646,6cyclictest1769784-21kworker/2:023:42:202
1706113995646,6cyclictest1705701-21kworker/2:321:12:022
1706113995645,6cyclictest1705701-21kworker/2:321:10:012
170610599561,9cyclictest1707616-21systemctl19:40:000
170611899552,17cyclictest1706445-21cat19:15:023
170611899552,17cyclictest1706445-21cat19:15:013
1706118995518,11cyclictest1799016-1kworker/u9:000:18:253
1706118995513,9cyclictest1788645-21grep23:50:013
1706118995513,9cyclictest1647312-1kworker/u9:121:31:503
1706113995546,6cyclictest1769784-21kworker/2:023:55:012
1706113995546,6cyclictest1769784-21kworker/2:023:55:012
1706113995546,5cyclictest1705701-21kworker/2:321:20:282
1706108995533,16cyclictest275-21dbus-daemon21:40:011
1706118995417,9cyclictest1805835-1kworker/u9:000:26:263
1706118995415,6cyclictest0-21swapper/300:00:293
170611899541,4cyclictest131rcu_preempt00:21:103
1706113995446,5cyclictest1769784-21kworker/2:023:30:062
1706113995446,5cyclictest1769784-21kworker/2:023:30:062
1706113995446,5cyclictest1752749-21kworker/2:222:58:392
1706113995446,5cyclictest1705701-21kworker/2:322:11:462
1706113995445,6cyclictest1709976-21kworker/2:021:42:032
1706113995444,6cyclictest1769784-21kworker/2:023:36:422
1706113995444,6cyclictest1709976-21kworker/2:022:01:512
170610599541,4cyclictest0-21swapper/020:54:480
1706118995318,5cyclictest1777112-21sh23:25:303
1706118995315,9cyclictest1748490-1kworker/u9:222:53:313
1706118995315,9cyclictest1748490-1kworker/u9:222:53:313
1706118995313,10cyclictest1799016-1kworker/u9:000:14:093
1706113995345,5cyclictest1705701-21kworker/2:322:28:512
1706113995345,5cyclictest1705701-21kworker/2:322:28:512
1706113995344,5cyclictest1709976-21kworker/2:022:22:542
170610599537,9cyclictest0-21swapper/019:10:020
170610599531,14cyclictest1706559-21modprobe19:20:000
170611899523,17cyclictest1808681-21ssh00:31:393
170611899522,18cyclictest1725053-21ssh21:36:333
1706118995219,10cyclictest1752173-21sh22:33:433
1706118995217,6cyclictest1767310-21cat23:05:023
1706118995217,6cyclictest1767310-21cat23:05:023
1706113995244,5cyclictest1769784-21kworker/2:023:59:042
1706113995244,5cyclictest1769784-21kworker/2:023:19:382
1706113995244,5cyclictest1769784-21kworker/2:000:21:212
1706113995244,5cyclictest1769784-21kworker/2:000:10:242
1706113995243,6cyclictest1769784-21kworker/2:023:26:092
1706113995243,5cyclictest1709976-21kworker/2:021:45:102
1706113995242,5cyclictest1705701-21kworker/2:321:00:012
1706113995234,13cyclictest275-21dbus-daemon20:55:012
1706113995144,5cyclictest1705701-21kworker/2:319:10:022
1706113995144,4cyclictest1709976-21kworker/2:022:07:272
1706113995143,5cyclictest1769784-21kworker/2:000:17:272
1706113995143,5cyclictest1705701-21kworker/2:322:49:502
1706113995142,6cyclictest1705701-21kworker/2:322:30:032
1706113995142,5cyclictest1769784-21kworker/2:000:37:262
1706113995141,5cyclictest1705701-21kworker/2:320:20:012
170610599511,30cyclictest0-21swapper/019:59:460
1706118995018,4cyclictest1708193-21latency_hist19:50:013
1706118995015,5cyclictest0-21swapper/300:05:563
1706113995042,5cyclictest1786321-21kworker/2:300:02:142
1706113995042,5cyclictest1769784-21kworker/2:000:07:022
1706113995042,5cyclictest1709976-21kworker/2:021:56:212
1706113995042,5cyclictest1709976-21kworker/2:021:53:232
1706113995042,5cyclictest1705701-21kworker/2:322:37:312
1706113995042,5cyclictest1705701-21kworker/2:322:16:492
1706113995042,4cyclictest1769784-21kworker/2:000:34:142
1706113995041,6cyclictest1752749-21kworker/2:223:10:412
1706113995041,6cyclictest1752749-21kworker/2:223:10:412
1706113995040,6cyclictest1752749-21kworker/2:223:02:562
1706113995040,6cyclictest1752749-21kworker/2:223:02:562
1706108995030,15cyclictest275-21dbus-daemon00:00:011
1706118994926,20cyclictest37-21ksoftirqd/319:25:023
170611899492,17cyclictest1789754-21ssh23:52:033
170611899492,17cyclictest1789754-21ssh23:52:033
170611899492,17cyclictest1758024-21ssh22:45:363
1706118994910,5cyclictest0-21swapper/321:30:013
1706113994942,4cyclictest1752749-21kworker/2:223:08:432
1706113994941,5cyclictest1769784-21kworker/2:000:25:362
1706113994940,6cyclictest1769784-21kworker/2:023:48:452
1706113994940,6cyclictest1705701-21kworker/2:320:05:022
1706113994937,8cyclictest1705701-21kworker/2:319:50:002
1706113994928,16cyclictest3467602-21systemd-journal22:45:002
1706105994931,12cyclictest3467602-21systemd-journal19:45:010
1706105994926,4cyclictest0-21swapper/019:12:160
1706105994926,4cyclictest0-21swapper/019:12:160
170611399485,36cyclictest1712283-21diskmemload22:55:022
170611399485,36cyclictest1712283-21diskmemload22:55:022
1706113994839,6cyclictest1705701-21kworker/2:321:18:582
170611399482,8cyclictest1708650-21munin-run20:00:012
1706108994831,13cyclictest275-21dbus-daemon19:20:001
170611899479,12cyclictest309-21in:imuxsock20:20:003
1706113994738,5cyclictest1705701-21kworker/2:320:15:012
1706108994730,12cyclictest275-21dbus-daemon20:30:001
1706108994720,20cyclictest23-21ksoftirqd/100:08:431
170611899462,16cyclictest3467602-21systemd-journal19:45:013
1706113994627,13cyclictest3467602-21systemd-journal20:40:012
1706113994627,13cyclictest3467602-21systemd-journal20:40:012
1706108994631,10cyclictest3467602-21systemd-journal20:10:031
1706108994618,5cyclictest23-21ksoftirqd/122:37:381
1706118994533,8cyclictest1-21systemd19:40:013
1706113994537,5cyclictest1705701-21kworker/2:319:31:562
1706113994536,5cyclictest1705701-21kworker/2:319:12:542
1706113994536,5cyclictest1705701-21kworker/2:319:12:532
1706108994535,7cyclictest23-21ksoftirqd/100:37:001
1706108994532,10cyclictest23-21ksoftirqd/100:15:001
1706108994531,10cyclictest1-21systemd21:45:011
1706108994530,11cyclictest3467602-21systemd-journal21:05:011
1706108994529,13cyclictest1-21systemd23:35:021
1706108994529,13cyclictest1-21systemd23:35:011
1706108994528,12cyclictest275-21dbus-daemon00:05:011
1706108994527,13cyclictest275-21dbus-daemon20:40:001
1706108994527,13cyclictest275-21dbus-daemon20:40:001
170610599455,22cyclictest3467602-21systemd-journal19:55:020
170611899443,12cyclictest0-21swapper/320:10:003
1706113994436,5cyclictest1705701-21kworker/2:319:40:022
1706113994429,12cyclictest1-21systemd20:25:012
1706113994429,12cyclictest1-21systemd20:25:012
1706108994423,14cyclictest275-21dbus-daemon00:30:011
1706108994415,23cyclictest1776706-21latency_hist23:25:011
1706108994415,23cyclictest1776706-21latency_hist23:25:011
1706118994327,10cyclictest275-21dbus-daemon19:55:003
1706108994334,5cyclictest23-21ksoftirqd/123:40:001
1706108994334,5cyclictest1721760-21latency_hist21:30:001
1706108994332,8cyclictest1-21systemd20:20:011
1706108994331,9cyclictest3467602-21systemd-journal21:50:021
1706108994329,6cyclictest1717022-21modprobe21:20:001
1706108994327,13cyclictest1724366-21sed21:35:021
1706108994327,11cyclictest1748180-21sort22:25:021
1706108994313,20cyclictest1750411-21ssh22:30:021
1706108994313,20cyclictest1750411-21ssh22:30:021
170611899423,4cyclictest131rcu_preempt20:40:003
170611899423,4cyclictest131rcu_preempt20:40:003
1706108994233,7cyclictest1757512-21cron22:45:001
1706108994228,10cyclictest275-21dbus-daemon23:30:001
1706108994227,11cyclictest3467602-21systemd-journal22:45:021
1706108994222,15cyclictest275-21dbus-daemon20:25:011
1706108994222,15cyclictest275-21dbus-daemon20:25:001
1706108994214,7cyclictest221rcuc/119:40:011
170611899414,3cyclictest131rcu_preempt19:20:013
1706113994132,5cyclictest1705701-21kworker/2:319:16:282
1706108994131,7cyclictest23-21ksoftirqd/121:46:471
1706108994126,11cyclictest1711798-21systemd-run21:00:001
1706108994119,4cyclictest23-21ksoftirqd/123:12:011
1706108994119,4cyclictest23-21ksoftirqd/123:12:011
1706108994117,5cyclictest23-21ksoftirqd/122:16:241
170610899411,4cyclictest131rcu_preempt23:42:271
170610899411,4cyclictest131rcu_preempt23:42:261
170610599413,33cyclictest3467602-21systemd-journal19:34:590
1706105994132,6cyclictest19350irq/34-mmc120:47:580
1706105994131,8cyclictest12-21ksoftirqd/019:25:010
170611899406,9cyclictest37-21ksoftirqd/319:30:013
170611899406,9cyclictest37-21ksoftirqd/319:30:003
1706118994028,9cyclictest1733626-21ssh21:55:013
1706118994025,10cyclictest3467602-21systemd-journal20:15:003
1706113994032,5cyclictest1705701-21kworker/2:319:45:012
1706113994031,6cyclictest1705701-21kworker/2:319:25:022
1706113994030,6cyclictest1705701-21kworker/2:320:10:012
170610899406,5cyclictest131rcu_preempt21:12:221
170610899405,7cyclictest23-21ksoftirqd/123:10:021
1706108994031,5cyclictest1719415-21latency_hist21:25:011
170610899402,7cyclictest0-21swapper/123:53:451
170610899402,7cyclictest0-21swapper/123:53:451
1706108994026,10cyclictest3467602-21systemd-journal20:55:001
170611899394,4cyclictest131rcu_preempt21:10:033
1706118993928,9cyclictest37-21ksoftirqd/323:15:023
1706118993928,9cyclictest37-21ksoftirqd/323:15:023
170611899392,4cyclictest131rcu_preempt19:35:003
1706108993929,7cyclictest23-21ksoftirqd/119:35:011
1706108993928,8cyclictest1710972-21sshd20:41:551
1706108993926,8cyclictest309-21in:imuxsock19:15:011
1706108993926,8cyclictest309-21in:imuxsock19:15:011
1706108993920,15cyclictest275-21dbus-daemon00:20:021
1706108993910,21cyclictest131rcu_preempt22:34:051
1706105993933,4cyclictest16950irq/224-800000020:58:560
170610599392,6cyclictest1709727-21unin-run20:20:000
1706118993816,6cyclictest37-21ksoftirqd/320:45:433
1706113993830,5cyclictest1705701-21kworker/2:320:30:012
1706113993829,5cyclictest1705701-21kworker/2:321:01:292
170610899383,5cyclictest131rcu_preempt23:45:471
1706108993827,8cyclictest23-21ksoftirqd/122:00:481
1706108993825,10cyclictest3467602-21systemd-journal23:03:321
1706108993825,10cyclictest3467602-21systemd-journal23:03:321
1706108993823,11cyclictest275-21dbus-daemon19:50:001
170610899381,33cyclictest48-21kcompactd022:05:301
1706105993823,11cyclictest1709468-21sed20:15:010
170610599381,33cyclictest0-21swapper/019:26:480
170610599381,33cyclictest0-21swapper/019:26:480
1706118993729,5cyclictest1711796-21sh21:00:013
170611899371,3cyclictest131rcu_preempt20:05:023
1706113993723,9cyclictest1708388-21munin-plugin-st19:55:002
1706113993717,14cyclictest3467602-21systemd-journal20:35:002
170610899374,25cyclictest1709171-21munin-plugin-st20:10:001
1706108993728,7cyclictest23-21ksoftirqd/122:14:591
1706108993726,6cyclictest309-21in:imuxsock21:10:001
1706108993722,7cyclictest309-21in:imuxsock20:00:011
1706108993711,19cyclictest1810118-21modprobe00:35:001
170610599372,31cyclictest3467602-21systemd-journal20:22:240
170610599372,31cyclictest3467602-21systemd-journal20:22:240
1706108993630,4cyclictest1762217-21latency_hist22:55:001
1706108993630,4cyclictest1762217-21latency_hist22:55:001
1706108993624,8cyclictest3467602-21systemd-journal19:30:011
1706108993624,8cyclictest3467602-21systemd-journal19:30:001
1706108993624,8cyclictest1708060-21grep19:45:031
1706108993613,18cyclictest3467602-21systemd-journal00:25:001
170611899353,3cyclictest131rcu_preempt21:05:013
1706118993527,5cyclictest1710458-21cron20:35:003
170610899353,24cyclictest1764664-21cron23:00:001
1706108993525,7cyclictest1706814-21sshd19:20:561
1706108993524,8cyclictest23-21ksoftirqd/120:01:151
1706108993520,12cyclictest1708542-21cut19:55:021
1706105993518,12cyclictest275-21dbus-daemon20:10:010
170610599351,6cyclictest1711001-21munin-run20:45:000
1706118993424,7cyclictest1706146-21sed19:10:023
1706113993417,12cyclictest3467602-21systemd-journal19:25:022
1706113993417,12cyclictest3467602-21systemd-journal19:25:022
170610899343,26cyclictest1710472-21grep20:35:001
1706108993421,9cyclictest23-21ksoftirqd/120:49:261
170611899332,3cyclictest131rcu_preempt20:51:423
170611899331,4cyclictest131rcu_preempt20:45:003
1706113993326,4cyclictest1711250-21latency_hist20:50:022
1706113993022,5cyclictest1705701-21kworker/2:320:45:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional