You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 10:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu May 07, 2026 00:45:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226444899871,46cyclictest2296742-21ssh22:04:201
226444899852,43cyclictest2285936-21ssh21:41:401
226444899842,41cyclictest2317497-21ssh22:47:111
226444899802,40cyclictest2329536-21ssh23:12:141
226444899791,46cyclictest2270614-21diskmemload21:18:451
226444899791,41cyclictest2292047-21ssh21:54:391
226444899791,41cyclictest0-21swapper/123:30:171
226444899791,41cyclictest0-21swapper/123:30:161
226444899791,39cyclictest2266976-21munin-run20:00:001
226444899781,44cyclictest2348212-21ssh23:50:561
226444899781,44cyclictest2348212-21ssh23:50:561
226444899781,44cyclictest2307740-21ssh22:26:491
226444899781,43cyclictest2277410-21ssh21:24:101
226444899771,41cyclictest2364139-21ssh00:24:131
226444899761,41cyclictest0-21swapper/122:41:151
226444899761,41cyclictest0-21swapper/121:12:531
226444899751,39cyclictest2320938-21ssh22:54:321
226444899751,39cyclictest2269158-21cut20:40:021
226444899731,39cyclictest2282562-21ssh21:34:571
226444899731,38cyclictest2288277-21ssh21:46:251
2264452997142,12cyclictest2255000-21kworker/2:019:30:032
226444899712,64cyclictest2300094-21ssh22:10:571
226444899712,37cyclictest2365687-21ssh00:27:061
2264448997115,33cyclictest2325264-1kworker/u9:123:48:021
226444899711,42cyclictest2333797-21diskmemload23:20:571
2264452996944,9cyclictest2255000-21kworker/2:022:01:532
2264448996929,36cyclictest2310561-21sh22:32:461
2264445996922,26cyclictest16850irq/134-800000021:40:030
226444899682,62cyclictest2324815-21ssh23:02:201
226444899681,63cyclictest2298077-21ssh22:06:581
226444899681,61cyclictest2270329-21sshd21:00:511
226444899681,43cyclictest0-21swapper/123:39:401
2264452996742,9cyclictest2270137-21kworker/2:323:34:512
2264452996742,9cyclictest2270137-21kworker/2:323:34:512
2264452996742,9cyclictest2270137-21kworker/2:323:00:282
2264448996729,34cyclictest2368230-21sh00:32:271
226444899671,62cyclictest2352922-21ssh00:00:351
226444899671,39cyclictest0-21swapper/122:15:251
226444899661,61cyclictest2323384-21ssh22:59:431
226444899661,61cyclictest2282837-21ssh21:35:031
226444899661,39cyclictest2355354-21ssh00:05:371
226444899661,39cyclictest2355354-21ssh00:05:361
226444899661,38cyclictest0-21swapper/121:55:261
226444899661,38cyclictest0-21swapper/121:55:261
2264445996631,17cyclictest2314343-21ssh22:40:260
226444899651,60cyclictest2336828-21ssh23:27:181
226444899651,60cyclictest2328143-21ssh23:09:281
226444899651,60cyclictest2312881-21ssh22:37:341
226444899651,60cyclictest2306684-21ssh22:24:571
226444899651,60cyclictest2278233-21ssh21:25:351
226444899651,34cyclictest2331479-21ssh23:16:101
2264445996528,16cyclictest2290169-21ssh21:50:170
226444899642,57cyclictest2267978-21cron20:17:001
226444899642,57cyclictest2267978-21cron20:16:591
226444899641,60cyclictest2358976-21ssh00:13:311
226444899641,59cyclictest2370472-21ssh00:37:121
226444899641,59cyclictest2370472-21ssh00:37:121
226444899641,59cyclictest2360033-21ssh00:15:281
226444899641,59cyclictest2360033-21ssh00:15:271
226444899641,59cyclictest2351034-21ssh23:56:511
226444899641,59cyclictest2351034-21ssh23:56:501
226444899641,59cyclictest2343267-21ssh23:40:341
226444899641,59cyclictest2343267-21ssh23:40:341
226444899641,59cyclictest2270123-21munin-run21:00:011
226444899641,58cyclictest2266680-21sshd19:50:371
226444899611,54cyclictest48-21kcompactd019:13:401
226444899611,54cyclictest48-21kcompactd019:13:401
226444899602,54cyclictest2267843-21cat20:15:021
226444899602,53cyclictest2266426-21sshd19:45:351
226444899601,55cyclictest2267477-21sshd20:08:481
226444899601,55cyclictest2267477-21sshd20:08:471
226444899601,55cyclictest2265141-21sshd19:20:481
226444899601,54cyclictest48-21kcompactd020:33:341
2264445996029,15cyclictest2281462-21ssh21:32:210
2264455995917,6cyclictest37-21ksoftirqd/323:35:023
2264455995917,6cyclictest37-21ksoftirqd/323:35:013
226444899592,53cyclictest2265680-21munin-run19:35:001
226444899591,53cyclictest364-21snmpd21:08:451
226444899591,52cyclictest364-21snmpd19:41:261
2264445995937,17cyclictest2362073-21ssh00:20:000
2264445995937,17cyclictest2362073-21ssh00:20:000
226444899582,51cyclictest2264935-21munin-run19:20:001
226444899581,52cyclictest2268509-21sshd20:25:481
2264452995749,5cyclictest2342854-21kworker/2:100:40:012
2264452995749,5cyclictest2342854-21kworker/2:100:40:012
226444899571,52cyclictest364-21snmpd20:54:391
226444899571,51cyclictest48-21kcompactd020:24:411
2264445995727,14cyclictest2353416-21ssh00:01:470
2264445995726,15cyclictest2271372-21ssh21:11:220
2264445995721,32cyclictest12-21ksoftirqd/021:29:350
2264455995646,7cyclictest37-21ksoftirqd/300:20:013
2264455995646,7cyclictest37-21ksoftirqd/300:20:013
2264452995646,6cyclictest2342854-21kworker/2:100:00:032
2264452995644,6cyclictest2270137-21kworker/2:321:45:002
2264452995642,11cyclictest30-21ksoftirqd/222:00:022
2264452995642,11cyclictest30-21ksoftirqd/222:00:012
226444899561,51cyclictest2269540-21sshd20:45:481
226444899561,50cyclictest364-21snmpd20:43:291
2264455995537,14cyclictest1-21systemd21:50:013
2264452995546,6cyclictest2255000-21kworker/2:022:26:532
2264452995546,5cyclictest2270137-21kworker/2:321:34:592
226444899551,49cyclictest364-21snmpd19:35:121
226444599552,44cyclictest0-21swapper/023:09:040
2264452995446,5cyclictest2342854-21kworker/2:123:50:552
2264452995446,5cyclictest2342854-21kworker/2:123:50:542
2264452995445,6cyclictest2270137-21kworker/2:323:21:172
2264452995445,6cyclictest2255000-21kworker/2:022:22:372
2264452995445,5cyclictest2255000-21kworker/2:021:16:342
2264452995444,6cyclictest2270137-21kworker/2:323:28:192
2264445995427,12cyclictest2322230-21ssh22:56:560
2264445995425,14cyclictest2358089-21ssh00:11:300
2264445995420,19cyclictest2305194-21ssh22:21:310
2264445995414,18cyclictest12-21ksoftirqd/000:22:240
226444599541,3cyclictest131rcu_preempt22:47:140
2264452995346,4cyclictest2342854-21kworker/2:100:22:322
2264452995346,4cyclictest2270137-21kworker/2:322:30:092
2264452995345,5cyclictest2342854-21kworker/2:123:46:002
2264452995345,5cyclictest2270137-21kworker/2:322:53:432
2264452995345,5cyclictest2270137-21kworker/2:321:26:102
2264452995345,5cyclictest2270137-21kworker/2:321:22:582
2264452995345,5cyclictest2255000-21kworker/2:022:37:522
2264452995345,5cyclictest2255000-21kworker/2:022:16:192
2264452995344,5cyclictest2255000-21kworker/2:021:00:012
2264452995343,5cyclictest2270137-21kworker/2:323:10:182
226444899531,48cyclictest364-21snmpd19:26:171
226444899531,47cyclictest364-21snmpd20:01:281
2264445995327,11cyclictest2343526-21ssh23:41:060
2264445995327,11cyclictest2343526-21ssh23:41:060
2264445995325,12cyclictest2366189-21ssh00:28:210
2264445995325,12cyclictest2337863-21ssh23:29:380
2264445995313,7cyclictest12-21ksoftirqd/021:25:020
226444599531,10cyclictest2348649-21ssh23:51:520
226444599531,10cyclictest2348649-21ssh23:51:510
226445599524,6cyclictest131rcu_preempt21:15:013
2264452995244,5cyclictest2342854-21kworker/2:123:57:252
2264452995244,5cyclictest2342854-21kworker/2:123:57:242
2264452995244,5cyclictest2342854-21kworker/2:100:13:162
2264452995244,5cyclictest2270137-21kworker/2:321:46:332
2264452995244,5cyclictest2270137-21kworker/2:321:39:212
2264452995243,4cyclictest2270137-21kworker/2:323:44:302
2264452995243,4cyclictest2270137-21kworker/2:323:44:302
2264452995232,15cyclictest275-21dbus-daemon20:25:002
2264452995144,4cyclictest2270137-21kworker/2:323:19:042
2264452995143,5cyclictest2342854-21kworker/2:100:16:092
2264452995143,5cyclictest2342854-21kworker/2:100:16:082
2264452995143,5cyclictest2270137-21kworker/2:322:58:182
2264452995143,5cyclictest2270137-21kworker/2:322:49:502
2264452995143,5cyclictest2270137-21kworker/2:322:08:232
2264452995142,6cyclictest2255000-21kworker/2:021:01:242
2264452995134,14cyclictest1-21systemd23:39:592
2264445995133,12cyclictest3467602-21systemd-journal20:45:010
2264445995124,12cyclictest2289237-21ssh21:48:350
226444599511,41cyclictest2345999-21sh23:46:200
226444599511,28cyclictest131rcu_preempt23:12:250
2264452995042,5cyclictest2270137-21kworker/2:323:07:562
2264452995042,5cyclictest2255000-21kworker/2:020:45:012
2264452995041,6cyclictest2342854-21kworker/2:100:08:382
2264452995041,6cyclictest2342854-21kworker/2:100:08:372
2264445995028,7cyclictest2339822-21ssh23:33:400
2264445995028,7cyclictest2339822-21ssh23:33:400
226444599502,7cyclictest0-21swapper/022:34:470
2264445995023,12cyclictest2296939-21ssh22:04:460
2264445995022,12cyclictest2324959-21ssh23:02:370
2264445995021,13cyclictest2342856-21ssh23:40:010
226444599501,3cyclictest131rcu_preempt23:21:230
2264455994936,10cyclictest37-21ksoftirqd/319:40:013
2264452994942,4cyclictest2357258-21kworker/2:000:28:262
2264452994941,5cyclictest2357258-21kworker/2:000:30:372
2264452994941,5cyclictest2255000-21kworker/2:022:41:142
2264452994941,5cyclictest2255000-21kworker/2:021:10:582
2264452994939,6cyclictest2255000-21kworker/2:022:11:192
2264445994923,11cyclictest2300938-21ssh22:12:530
226444599492,27cyclictest131rcu_preempt21:17:080
2264445994914,31cyclictest0-21swapper/023:58:040
2264445994914,31cyclictest0-21swapper/023:58:040
226445599484,29cyclictest361rcuc/300:05:013
2264455994831,14cyclictest37-21ksoftirqd/323:26:523
2264455994814,8cyclictest131rcu_preempt22:29:123
2264452994840,5cyclictest2270137-21kworker/2:321:52:142
2264452994839,5cyclictest2255000-21kworker/2:020:14:482
226444599488,35cyclictest2369420-21latency_hist00:35:020
226444599483,16cyclictest2332213-21ssh23:17:500
2264445994814,31cyclictest12-21ksoftirqd/021:38:230
2264455994732,9cyclictest2266954-21cron20:00:003
2264455994730,13cyclictest37-21ksoftirqd/321:56:523
2264455994730,13cyclictest37-21ksoftirqd/321:56:513
2264455994729,12cyclictest1-21systemd21:05:013
2264452994739,5cyclictest2255000-21kworker/2:021:10:012
2264452994739,5cyclictest2255000-21kworker/2:019:13:402
2264452994739,5cyclictest2255000-21kworker/2:019:13:392
2264452994733,10cyclictest3467602-21systemd-journal20:00:012
2264455994634,10cyclictest37-21ksoftirqd/321:55:023
2264455994628,14cyclictest275-21dbus-daemon00:00:013
2264455994628,14cyclictest275-21dbus-daemon00:00:013
2264455994628,13cyclictest3467602-21systemd-journal22:40:003
2264455994624,15cyclictest275-21dbus-daemon00:25:003
2264445994634,8cyclictest12-21ksoftirqd/020:24:290
2264445994633,9cyclictest12-21ksoftirqd/021:00:010
2264445994628,15cyclictest2319607-21ssh22:51:280
2264445994628,14cyclictest2303469-21ssh22:18:090
2264445994626,4cyclictest2307612-21rm22:26:350
2264445994612,16cyclictest0-21swapper/021:59:510
2264445994612,16cyclictest0-21swapper/021:59:510
226445599459,5cyclictest131rcu_preempt21:25:483
226445599455,6cyclictest131rcu_preempt23:36:093
2264455994532,10cyclictest37-21ksoftirqd/323:53:433
2264455994532,10cyclictest37-21ksoftirqd/323:53:433
2264455994531,11cyclictest37-21ksoftirqd/319:45:013
2264455994530,11cyclictest3467602-21systemd-journal20:57:523
2264455994529,13cyclictest37-21ksoftirqd/320:05:033
2264455994528,11cyclictest37-21ksoftirqd/322:15:003
2264455994524,15cyclictest275-21dbus-daemon23:25:013
226445599451,9cyclictest37-21ksoftirqd/323:08:023
226445599451,8cyclictest0-21swapper/300:11:573
2264455994517,6cyclictest131rcu_preempt21:23:423
2264455994516,22cyclictest131rcu_preempt00:34:473
2264455994510,10cyclictest131rcu_preempt23:00:033
2264445994538,4cyclictest16950irq/224-800000020:15:000
2264445994536,6cyclictest16950irq/224-800000000:39:200
2264445994536,6cyclictest16950irq/224-800000000:39:200
2264445994528,13cyclictest2312816-21ssh22:37:260
2264445994528,13cyclictest2297928-21ssh22:06:260
226445599445,17cyclictest2282689-21ssh21:35:023
226445599444,8cyclictest37-21ksoftirqd/323:15:003
2264455994433,7cyclictest37-21ksoftirqd/300:10:013
2264455994433,7cyclictest37-21ksoftirqd/300:10:013
2264455994433,7cyclictest2294720-21sh22:00:033
2264455994420,20cyclictest37-21ksoftirqd/321:40:573
226445599441,7cyclictest0-21swapper/322:45:303
2264455994410,6cyclictest131rcu_preempt21:39:593
2264452994436,5cyclictest2255000-21kworker/2:019:35:392
2264452994435,6cyclictest2255000-21kworker/2:020:05:002
226445599436,6cyclictest131rcu_preempt22:52:023
2264455994332,8cyclictest37-21ksoftirqd/323:20:023
2264455994332,8cyclictest37-21ksoftirqd/322:18:143
2264455994325,13cyclictest3467602-21systemd-journal21:10:013
226445599431,5cyclictest0-21swapper/319:55:003
2264455994311,6cyclictest131rcu_preempt20:20:013
2264455994311,6cyclictest131rcu_preempt20:20:003
2264445994328,12cyclictest2355839-21ssh00:06:500
2264445994328,12cyclictest2355839-21ssh00:06:500
226444599432,12cyclictest2269055-21munin-run20:40:000
226444599431,19cyclictest150-21pr/ttyS219:50:020
2264445994311,8cyclictest12-21ksoftirqd/020:06:410
2264445994311,8cyclictest12-21ksoftirqd/020:06:400
226445599428,6cyclictest131rcu_preempt22:32:303
226445599425,5cyclictest131rcu_preempt23:45:003
226445599425,5cyclictest131rcu_preempt23:45:003
2264455994229,10cyclictest37-21ksoftirqd/322:24:583
2264455994213,6cyclictest131rcu_preempt19:25:013
2264452994234,5cyclictest2255000-21kworker/2:020:34:022
2264452994233,6cyclictest2255000-21kworker/2:019:51:392
226444899422,34cyclictest2264521-21cat19:10:011
2264445994235,4cyclictest16950irq/224-800000019:20:000
226444599422,7cyclictest2268970-21cat20:35:030
226445599415,7cyclictest37-21ksoftirqd/322:07:523
2264455994133,5cyclictest37-21ksoftirqd/321:18:123
2264455994129,8cyclictest37-21ksoftirqd/300:35:563
2264455994129,8cyclictest37-21ksoftirqd/300:35:553
2264455994129,8cyclictest2269564-21cron20:50:003
2264455994128,7cyclictest309-21in:imuxsock20:30:003
2264455994126,11cyclictest37-21ksoftirqd/320:15:013
2264455994112,5cyclictest131rcu_preempt22:41:273
2264452994134,5cyclictest2255000-21kworker/2:019:10:022
2264452994133,5cyclictest2255000-21kworker/2:019:35:012
2264452994132,6cyclictest2255000-21kworker/2:019:49:312
226444599412,6cyclictest2269591-21munin-run20:50:010
226444599411,13cyclictest3467602-21systemd-journal19:41:150
2264455994019,8cyclictest2268813-21cat20:35:023
2264455994016,20cyclictest131rcu_preempt00:25:563
2264452994032,5cyclictest2255000-21kworker/2:020:47:332
2264452994032,5cyclictest2255000-21kworker/2:020:40:002
2264452994026,11cyclictest30-21ksoftirqd/219:41:482
2264445994033,4cyclictest16950irq/224-800000019:15:020
2264445994033,4cyclictest16950irq/224-800000019:15:010
2264445994026,10cyclictest275-21dbus-daemon20:20:010
2264445994026,10cyclictest275-21dbus-daemon20:20:000
2264445994024,13cyclictest275-21dbus-daemon20:05:000
2264455993931,6cyclictest37-21ksoftirqd/320:40:023
226445599391,6cyclictest0-21swapper/323:04:273
2264452993931,5cyclictest2255000-21kworker/2:020:06:052
2264452993931,5cyclictest2255000-21kworker/2:020:06:052
2264445993929,6cyclictest3467602-21systemd-journal19:23:500
2264445993925,11cyclictest12-21ksoftirqd/021:10:020
2264445993924,11cyclictest275-21dbus-daemon20:30:010
226445599384,10cyclictest37-21ksoftirqd/323:45:423
2264455993830,6cyclictest37-21ksoftirqd/319:12:093
2264455993830,6cyclictest37-21ksoftirqd/319:12:083
2264455993825,10cyclictest275-21dbus-daemon20:55:013
226445299382,6cyclictest0-21swapper/220:30:022
2264445993827,8cyclictest2265906-21sshd19:37:470
226444599381,3cyclictest131rcu_preempt20:55:000
226444599381,33cyclictest0-21swapper/019:30:020
2264455993726,8cyclictest37-21ksoftirqd/319:15:473
2264445993722,11cyclictest3467602-21systemd-journal20:00:010
2264455993624,8cyclictest37-21ksoftirqd/319:50:013
2264455993624,7cyclictest309-21in:imuxsock19:30:003
2264452993627,6cyclictest2255000-21kworker/2:019:21:122
2264452993627,6cyclictest2255000-21kworker/2:019:19:472
2264445993625,9cyclictest12-21ksoftirqd/019:35:020
2264445993623,10cyclictest12-21ksoftirqd/021:00:280
226444599361,31cyclictest0-21swapper/019:53:210
226445599357,6cyclictest131rcu_preempt19:35:013
226445599353,8cyclictest37-21ksoftirqd/320:21:483
2264455993523,9cyclictest37-21ksoftirqd/320:45:013
2264452993527,5cyclictest2255000-21kworker/2:020:53:512
2264455993422,9cyclictest2264498-21sed19:10:013
226445599334,5cyclictest131rcu_preempt20:05:233
226445599334,5cyclictest131rcu_preempt20:05:233
226445299321,5cyclictest131rcu_preempt20:20:012
226445299321,5cyclictest131rcu_preempt20:20:012
226444599287,13cyclictest2264443-21ls19:10:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional