You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 18:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sat Jun 27, 2026 12:45:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
208140699117113,2cyclictest0-21swapper/012:15:020
2081406999382,8cyclictest16950irq/224-800000010:20:010
2081406999057,29cyclictest16850irq/134-800000009:20:020
2081406998442,37cyclictest2123164-21munin-run10:25:010
2081406998442,37cyclictest2123164-21munin-run10:25:010
2081406998428,15cyclictest2187039-21munin-run12:40:000
208140699821,77cyclictest0-21swapper/007:40:030
2081416998116,61cyclictest2067954-21munin-node07:20:023
2081410997710,49cyclictest2097231-21ssh09:30:031
2081406997728,13cyclictest2151466-21munin-plugin-st11:25:000
2081406997523,48cyclictest2126502-21ssh10:31:510
2081406997421,17cyclictest2158504-21munin-run11:40:000
2081413997320,14cyclictest3467602-21systemd-journal09:40:342
2081406997322,43cyclictest2086707-21latency_hist08:50:020
2081406997222,16cyclictest2110128-21ssh09:57:340
2081406997214,52cyclictest0-21swapper/011:30:300
2081406997119,35cyclictest2117113-21ssh10:12:090
2081406997025,13cyclictest2167716-21ssh11:59:170
2081406997020,15cyclictest2176938-21ssh12:18:320
2081406997017,17cyclictest2106506-21munin-run09:50:010
2081406996920,15cyclictest1-21systemd11:51:350
2081416996821,11cyclictest37-21ksoftirqd/308:10:023
2081406996813,41cyclictest19350irq/34-mmc111:10:020
2081406996812,28cyclictest2136371-21ssh10:53:050
2081413996727,11cyclictest309-21in:imuxsock07:40:012
208140699676,31cyclictest0-21swapper/011:11:060
2081413996631,31cyclictest2087623-21diskmemload09:30:342
2081406996632,5cyclictest2134859-21munin-run10:50:010
2081406996632,30cyclictest2182673-21ssh12:30:200
2081406996624,15cyclictest2087130-21unin-run09:00:010
2081406996624,15cyclictest2087130-21unin-run09:00:010
2081406996623,17cyclictest2152657-21sh11:27:130
2081406996622,21cyclictest0-21swapper/009:03:180
2081413996519,29cyclictest2082855-21kworker/u8:307:45:022
2081406996534,7cyclictest2085564-21munin-run08:30:010
2081406996515,33cyclictest2140211-21ssh11:00:580
2081413996423,7cyclictest2106617-21latency_hist09:50:032
2081406996427,33cyclictest3467602-21systemd-journal09:05:020
2081413996325,5cyclictest2090777-21sh09:16:322
2081413996320,8cyclictest2087623-21diskmemload10:25:022
2081413996320,8cyclictest2087623-21diskmemload10:25:022
208140699638,25cyclictest0-21swapper/011:47:370
208140699638,25cyclictest0-21swapper/011:47:370
2081406996321,9cyclictest2113273-21ssh10:04:160
2081406996321,9cyclictest2113273-21ssh10:04:160
2081406996317,41cyclictest2095808-21ssh09:27:070
2081406996315,20cyclictest0-21swapper/009:33:290
2081413996227,8cyclictest30-21ksoftirqd/209:24:062
2081413996218,10cyclictest275-21dbus-daemon08:00:002
2081406996218,12cyclictest2099866-21rm09:35:390
2081406996217,41cyclictest2128933-21ssh10:37:120
2081406996213,12cyclictest0-21swapper/010:43:570
2081413996025,4cyclictest2145323-21sh11:11:512
2081413996024,5cyclictest2184747-21latency_hist12:35:022
2081413996016,13cyclictest1-21systemd07:30:002
2081406996025,4cyclictest2115975-21munin-run10:10:010
2081406996021,35cyclictest2085279-21sshd08:23:350
208141099593,41cyclictest221rcuc/108:15:011
2081406995939,16cyclictest16850irq/134-800000011:45:000
2081406995923,8cyclictest2179981-21unin-run12:25:000
2081413995824,5cyclictest2128239-21sh10:35:332
2081413995824,5cyclictest2109596-21sh09:56:152
2081406995842,13cyclictest2084421-21sshd08:05:140
2081406995830,23cyclictest2139585-21cron11:00:000
2081406995821,9cyclictest2084187-21modprobe08:03:340
2081406995818,34cyclictest0-21swapper/012:10:030
2081416995737,15cyclictest3467602-21systemd-journal08:00:003
2081413995724,8cyclictest30-21ksoftirqd/209:12:112
2081410995729,22cyclictest23-21ksoftirqd/107:40:021
2081406995721,31cyclictest364-21snmpd07:36:300
2081406995720,11cyclictest0-21swapper/009:21:410
2081413995636,13cyclictest275-21dbus-daemon11:50:012
2081413995636,13cyclictest275-21dbus-daemon11:50:012
2081413995623,8cyclictest30-21ksoftirqd/209:53:142
2081413995623,8cyclictest30-21ksoftirqd/209:53:142
2081413995623,8cyclictest2135333-21sshd10:50:342
2081413995621,10cyclictest30-21ksoftirqd/212:19:352
2081413995620,10cyclictest30-21ksoftirqd/212:06:212
2081413995619,9cyclictest30-21ksoftirqd/210:43:352
2081406995641,11cyclictest2084762-21systemd-run08:15:000
2081406995640,13cyclictest2086030-21sshd08:37:340
2081406995637,15cyclictest2083396-21sshd07:47:340
2081406995631,19cyclictest2084981-21sshd08:16:340
2081406995620,32cyclictest0-21swapper/012:26:570
2081406995616,15cyclictest2168855-21ssh12:01:170
2081413995525,5cyclictest2157461-21sh11:37:322
2081413995523,9cyclictest2086542-21sed08:45:032
2081413995523,5cyclictest2086834-21latency_hist08:55:022
208141099553,49cyclictest2092339-21ssh09:20:021
2081410995534,15cyclictest3467602-21systemd-journal11:35:541
2081406995541,11cyclictest2082638-21munin-run07:35:000
2081413995446,5cyclictest2137237-21kworker/2:311:30:152
2081413995445,5cyclictest2082944-21kworker/2:307:45:012
2081413995427,5cyclictest2139719-21latency_hist11:00:032
2081413995426,7cyclictest30-21ksoftirqd/212:11:252
2081413995423,11cyclictest30-21ksoftirqd/212:39:082
2081413995422,12cyclictest3467602-21systemd-journal11:05:002
2081413995420,11cyclictest30-21ksoftirqd/210:30:002
2081406995441,10cyclictest2082115-21sshd07:23:340
2081406995441,10cyclictest2082115-21sshd07:23:340
2081406995432,18cyclictest2148433-21sshd11:18:340
2081406995428,22cyclictest0-21swapper/007:55:030
2081406995421,29cyclictest2102545-21ssh09:41:120
2081406995415,8cyclictest2108328-21sh09:53:440
2081406995415,8cyclictest2108328-21sh09:53:440
2081416995335,14cyclictest1-21systemd07:35:013
2081413995325,7cyclictest30-21ksoftirqd/209:26:352
2081406995337,12cyclictest2081839-21sshd07:19:340
2081406995335,14cyclictest2086821-21sshd08:54:340
2081406995332,16cyclictest364-21snmpd07:13:230
2081413995244,5cyclictest2082944-21kworker/2:308:30:002
2081413995243,6cyclictest2086615-21kworker/2:010:16:212
2081413995227,4cyclictest2115956-21latency_hist10:10:012
2081406995240,9cyclictest2124404-21sshd10:27:330
2081406995239,10cyclictest2083896-21sshd07:56:340
2081406995237,12cyclictest2086310-21sshd08:42:340
2081413995142,6cyclictest2127871-21kworker/2:110:49:422
2081413995142,5cyclictest2081385-21kworker/2:211:45:022
2081413995123,5cyclictest2117902-21sh10:13:572
208141399512,12cyclictest2165675-21ssh11:55:002
2081406995142,5cyclictest2082399-21munin-run07:30:010
2081406995140,9cyclictest2085774-21sshd08:31:340
2081406995140,9cyclictest2085774-21sshd08:31:340
2081406995139,9cyclictest2083640-21sshd07:50:340
2081406995123,13cyclictest2088875-21ssh09:12:420
2081416994914,27cyclictest2086550-21kworker/u8:308:50:023
2081416994913,9cyclictest131rcu_preempt10:45:003
2081413994943,3cyclictest2081385-21kworker/2:210:31:212
2081410994932,13cyclictest3467602-21systemd-journal07:25:011
2081410994932,13cyclictest3467602-21systemd-journal07:25:011
2081410994922,21cyclictest23-21ksoftirqd/111:15:001
2081416994836,7cyclictest37-21ksoftirqd/311:25:013
2081416994828,14cyclictest1-21systemd10:10:003
2081413994842,4cyclictest2081385-21kworker/2:210:02:222
2081413994842,4cyclictest2081385-21kworker/2:210:02:222
2081413994841,5cyclictest2137237-21kworker/2:311:08:102
2081413994839,5cyclictest2081385-21kworker/2:211:16:272
2081410994832,11cyclictest2083910-21cron08:00:011
2081410994830,12cyclictest275-21dbus-daemon10:40:021
2081410994823,21cyclictest23-21ksoftirqd/111:44:381
2081410994820,22cyclictest23-21ksoftirqd/111:49:441
2081410994820,22cyclictest23-21ksoftirqd/111:49:441
2081416994734,9cyclictest37-21ksoftirqd/307:50:023
2081416994717,21cyclictest37-21ksoftirqd/311:54:523
2081413994739,5cyclictest2137237-21kworker/2:311:26:022
2081413994736,7cyclictest2172859-21kworker/2:012:24:072
2081413994732,11cyclictest2087707-21latency_hist09:10:022
2081413994713,15cyclictest2149040-21kworker/u8:411:25:032
2081410994729,14cyclictest3467602-21systemd-journal09:20:021
2081410994722,20cyclictest23-21ksoftirqd/110:05:571
2081416994632,10cyclictest3467602-21systemd-journal12:37:353
208141699462,4cyclictest131rcu_preempt11:35:403
2081416994619,19cyclictest37-21ksoftirqd/312:08:463
2081416994617,21cyclictest37-21ksoftirqd/310:53:393
2081416994613,8cyclictest131rcu_preempt10:31:213
2081413994638,5cyclictest2086615-21kworker/2:009:37:492
2081413994628,14cyclictest275-21dbus-daemon07:15:012
2081410994622,20cyclictest23-21ksoftirqd/111:15:201
2081416994536,6cyclictest37-21ksoftirqd/311:30:013
2081416994530,12cyclictest37-21ksoftirqd/309:16:563
2081416994517,20cyclictest37-21ksoftirqd/311:04:393
2081416994514,9cyclictest131rcu_preempt09:29:373
2081410994528,12cyclictest275-21dbus-daemon09:15:031
2081410994527,13cyclictest275-21dbus-daemon09:45:011
2081410994521,21cyclictest23-21ksoftirqd/109:54:531
2081410994521,21cyclictest23-21ksoftirqd/109:54:531
2081410994520,20cyclictest23-21ksoftirqd/112:38:551
208141699448,5cyclictest131rcu_preempt11:18:233
208141699447,7cyclictest131rcu_preempt11:10:023
2081416994435,6cyclictest37-21ksoftirqd/309:30:053
2081416994431,9cyclictest364-21snmpd12:04:273
2081416994417,19cyclictest37-21ksoftirqd/311:45:103
2081416994417,19cyclictest37-21ksoftirqd/311:45:093
2081416994417,19cyclictest37-21ksoftirqd/309:44:443
2081416994410,10cyclictest131rcu_preempt10:25:033
2081416994410,10cyclictest131rcu_preempt10:25:023
2081413994436,5cyclictest2081385-21kworker/2:212:00:592
2081413994428,12cyclictest275-21dbus-daemon08:40:002
2081413994426,13cyclictest0-21swapper/209:00:002
2081413994426,13cyclictest0-21swapper/209:00:002
2081413994414,7cyclictest131rcu_preempt11:56:062
2081410994427,13cyclictest275-21dbus-daemon09:35:011
2081410994427,13cyclictest23-21ksoftirqd/110:05:011
2081410994427,13cyclictest23-21ksoftirqd/110:05:011
2081410994421,19cyclictest23-21ksoftirqd/112:25:291
2081410994419,21cyclictest23-21ksoftirqd/111:02:061
2081410994419,20cyclictest23-21ksoftirqd/110:50:411
2081410994418,21cyclictest23-21ksoftirqd/111:24:501
2081416994332,8cyclictest1-21systemd09:25:003
2081416994331,10cyclictest37-21ksoftirqd/310:11:013
2081416994326,12cyclictest275-21dbus-daemon07:55:013
2081416994326,12cyclictest275-21dbus-daemon07:55:013
2081416994313,8cyclictest131rcu_preempt11:34:303
2081416994313,5cyclictest131rcu_preempt10:15:123
208141699431,10cyclictest0-21swapper/309:45:323
2081416994310,8cyclictest131rcu_preempt09:53:233
2081416994310,8cyclictest131rcu_preempt09:53:233
2081410994334,6cyclictest23-21ksoftirqd/110:25:491
2081410994328,11cyclictest2144294-21cron11:10:001
2081410994327,12cyclictest2165848-21sed11:55:021
2081410994323,15cyclictest275-21dbus-daemon10:50:011
2081410994317,22cyclictest23-21ksoftirqd/112:10:001
2081410994314,19cyclictest23-21ksoftirqd/108:35:011
2081410994314,19cyclictest23-21ksoftirqd/108:35:011
2081416994227,11cyclictest2184661-21munin-run12:35:013
208141699422,6cyclictest131rcu_preempt10:50:023
2081416994217,20cyclictest275-21dbus-daemon10:05:013
2081416994217,20cyclictest275-21dbus-daemon10:05:013
208141699421,5cyclictest131rcu_preempt11:13:273
208141699421,36cyclictest37-21ksoftirqd/311:42:383
2081416994212,21cyclictest361rcuc/312:14:203
2081413994233,6cyclictest2137237-21kworker/2:312:27:572
2081413994227,11cyclictest2082143-21systemd-run07:25:012
2081413994227,11cyclictest2082143-21systemd-run07:25:012
2081410994233,7cyclictest23-21ksoftirqd/110:14:491
2081410994233,5cyclictest2101765-21cron09:40:001
2081410994232,6cyclictest2179984-21systemd12:25:011
2081410994230,9cyclictest23-21ksoftirqd/108:25:021
2081410994227,12cyclictest23-21ksoftirqd/110:17:001
2081410994224,12cyclictest23-21ksoftirqd/109:50:011
2081410994222,14cyclictest275-21dbus-daemon08:20:011
2081410994219,5cyclictest23-21ksoftirqd/112:16:261
208141699419,6cyclictest131rcu_preempt09:55:343
2081416994129,9cyclictest37-21ksoftirqd/308:30:023
2081416994127,11cyclictest37-21ksoftirqd/312:23:443
2081413994131,6cyclictest30-21ksoftirqd/208:12:152
2081413994130,7cyclictest275-21dbus-daemon08:20:012
2081410994128,9cyclictest275-21dbus-daemon07:35:011
2081410994125,11cyclictest23-21ksoftirqd/108:40:001
2081410994124,12cyclictest275-21dbus-daemon07:50:011
2081410994123,14cyclictest3467602-21systemd-journal07:42:341
2081410994112,7cyclictest131rcu_preempt08:10:021
208141699404,11cyclictest37-21ksoftirqd/312:19:453
2081416994029,8cyclictest37-21ksoftirqd/311:00:023
2081416994029,7cyclictest37-21ksoftirqd/309:12:333
2081416994026,5cyclictest131rcu_preempt08:45:023
208141699401,8cyclictest0-21swapper/309:37:293
208141699401,7cyclictest0-21swapper/310:37:073
2081413994030,6cyclictest309-21in:imuxsock07:20:002
2081410994029,9cyclictest23-21ksoftirqd/111:57:021
2081410994028,10cyclictest23-21ksoftirqd/110:56:431
208141099401,3cyclictest131rcu_preempt10:40:131
2081406994018,16cyclictest2081483-21latency_hist07:10:020
208141699399,6cyclictest131rcu_preempt12:29:373
2081416993927,10cyclictest275-21dbus-daemon09:00:013
2081416993927,10cyclictest275-21dbus-daemon09:00:013
2081416993926,10cyclictest2084200-21cron08:05:003
2081416993911,5cyclictest131rcu_preempt08:40:003
208141099394,4cyclictest131rcu_preempt12:11:511
208141099393,5cyclictest131rcu_preempt07:10:021
208141099392,3cyclictest131rcu_preempt12:00:281
2081410993916,19cyclictest23-21ksoftirqd/109:55:091
2081410993916,15cyclictest23-21ksoftirqd/110:34:291
208141699389,6cyclictest131rcu_preempt10:26:093
2081416993817,15cyclictest275-21dbus-daemon07:45:003
208141699381,5cyclictest0-21swapper/311:55:433
2081413993825,9cyclictest3467602-21systemd-journal08:08:342
208141099387,28cyclictest2081136-21kworker/u8:107:15:021
2081410993828,5cyclictest23-21ksoftirqd/111:26:521
2081410993826,8cyclictest2156403-21latency_hist11:35:021
208141699379,5cyclictest131rcu_preempt08:10:153
208141699373,10cyclictest37-21ksoftirqd/307:30:003
2081416993729,5cyclictest2087606-21cron09:10:003
2081416993727,7cyclictest37-21ksoftirqd/308:35:013
2081416993727,7cyclictest37-21ksoftirqd/308:35:013
2081416993722,11cyclictest37-21ksoftirqd/307:15:023
208141399375,6cyclictest131rcu_preempt07:55:002
208141399375,6cyclictest131rcu_preempt07:55:002
2081410993729,4cyclictest23-21ksoftirqd/110:25:011
2081410993729,4cyclictest23-21ksoftirqd/110:25:011
2081410993721,7cyclictest23-21ksoftirqd/109:05:481
2081410993719,13cyclictest309-21in:imuxsock07:15:031
2081416993621,11cyclictest3467602-21systemd-journal07:40:003
2081416993619,13cyclictest2086850-21munin-run08:55:013
2081416993619,12cyclictest275-21dbus-daemon07:25:013
2081416993619,12cyclictest275-21dbus-daemon07:25:013
2081413993627,7cyclictest30-21ksoftirqd/207:10:022
2081413993624,9cyclictest30-21ksoftirqd/209:05:002
2081413993622,11cyclictest2084413-21sed08:05:022
208141399361,9cyclictest2085424-21cat08:25:022
208141099368,9cyclictest23-21ksoftirqd/109:00:021
208141099368,9cyclictest23-21ksoftirqd/109:00:011
2081410993611,17cyclictest23-21ksoftirqd/112:31:201
208141699358,5cyclictest131rcu_preempt08:25:013
2081416993510,5cyclictest131rcu_preempt08:20:023
2081410993421,9cyclictest1-21systemd08:50:011
2081413993322,9cyclictest30-21ksoftirqd/207:35:022
208141099335,21cyclictest131rcu_preempt08:30:011
2081410993326,4cyclictest2084203-21latency_hist08:05:011
2081410993324,5cyclictest23-21ksoftirqd/108:45:011
2081410993323,5cyclictest2082385-21munin-plugin-st07:30:001
2081410993322,8cyclictest23-21ksoftirqd/109:05:021
2081416993220,9cyclictest2081465-21sed07:10:013
208141699321,6cyclictest0-21swapper/309:01:083
2081413993218,10cyclictest30-21ksoftirqd/208:33:042
2081413993218,10cyclictest30-21ksoftirqd/208:33:042
2081410993219,10cyclictest275-21dbus-daemon07:55:011
2081410993219,10cyclictest275-21dbus-daemon07:55:011
2081413993014,12cyclictest3467602-21systemd-journal08:45:352
208141099297,9cyclictest23-21ksoftirqd/108:50:451
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional