You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 17:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Jul 12, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28093149910749,8cyclictest131rcu_preempt10:00:031
2809311999489,2cyclictest0-21swapper/010:20:020
2809311999452,8cyclictest309-21in:imuxsock11:00:000
280931199931,88cyclictest0-21swapper/008:50:020
2809311997611,45cyclictest16950irq/224-800000009:15:020
2809311997043,24cyclictest0-21swapper/010:00:030
2809325996718,10cyclictest37-21ksoftirqd/307:25:013
2809325996715,11cyclictest37-21ksoftirqd/312:01:353
2809325996617,12cyclictest37-21ksoftirqd/312:05:353
2809325996616,11cyclictest37-21ksoftirqd/309:11:463
2809325996614,12cyclictest37-21ksoftirqd/311:44:303
2809325996416,11cyclictest37-21ksoftirqd/308:40:013
2809325996415,10cyclictest37-21ksoftirqd/310:50:293
2809325996414,12cyclictest37-21ksoftirqd/309:10:023
280932599636,19cyclictest2910631-21ssh12:34:113
2809325996316,10cyclictest37-21ksoftirqd/309:51:353
2809325996316,10cyclictest37-21ksoftirqd/309:51:343
280932599625,18cyclictest2888820-21ssh11:47:293
2809325996215,13cyclictest37-21ksoftirqd/312:15:383
2809325996213,12cyclictest37-21ksoftirqd/311:37:433
2809325996213,10cyclictest37-21ksoftirqd/311:13:103
2809325996115,9cyclictest37-21ksoftirqd/311:15:413
2809325996114,9cyclictest37-21ksoftirqd/312:24:323
2809325996114,9cyclictest37-21ksoftirqd/310:58:003
2809314996028,9cyclictest23-21ksoftirqd/111:21:551
2809325995915,11cyclictest37-21ksoftirqd/309:42:533
2809325995914,8cyclictest37-21ksoftirqd/309:29:533
2809325995913,8cyclictest37-21ksoftirqd/310:24:003
2809325995913,11cyclictest37-21ksoftirqd/311:21:243
2809325995913,10cyclictest37-21ksoftirqd/310:46:473
2809325995913,10cyclictest37-21ksoftirqd/310:46:463
2809325995912,11cyclictest37-21ksoftirqd/309:36:353
2809322995944,7cyclictest2899245-21kworker/2:212:17:122
2809311995924,16cyclictest2820448-21ssh09:20:090
2809311995924,15cyclictest131rcu_preempt11:48:370
280932599585,18cyclictest2810821-21munin-run07:40:003
2809325995819,10cyclictest37-21ksoftirqd/310:05:003
2809325995815,10cyclictest37-21ksoftirqd/309:05:003
2809325995814,9cyclictest37-21ksoftirqd/311:26:253
2809325995813,8cyclictest37-21ksoftirqd/312:38:233
2809325995813,8cyclictest37-21ksoftirqd/312:38:233
2809325995813,11cyclictest37-21ksoftirqd/310:29:163
2809325995813,10cyclictest37-21ksoftirqd/310:41:593
2809325995813,10cyclictest37-21ksoftirqd/309:45:433
2809325995812,8cyclictest37-21ksoftirqd/310:08:223
2809325995812,8cyclictest37-21ksoftirqd/310:08:213
2809325995812,10cyclictest37-21ksoftirqd/309:17:003
2809325995811,10cyclictest37-21ksoftirqd/310:10:473
2809314995818,8cyclictest23-21ksoftirqd/110:52:381
2809325995714,8cyclictest37-21ksoftirqd/307:55:013
2809325995711,7cyclictest2908100-21ssh12:28:363
2809325995711,7cyclictest2908100-21ssh12:28:353
2809325995711,11cyclictest37-21ksoftirqd/311:03:053
2809325995711,10cyclictest37-21ksoftirqd/309:21:243
2809311995747,5cyclictest16950irq/224-800000011:20:000
2809311995727,14cyclictest2861305-21ssh10:48:140
2809311995727,14cyclictest2861305-21ssh10:48:130
2809311995726,15cyclictest2901848-21ssh12:15:050
280931199571,49cyclictest2831701-21latency_hist09:45:020
280932599564,15cyclictest2856285-21ssh10:37:193
2809325995617,7cyclictest37-21ksoftirqd/309:00:003
2809325995616,10cyclictest37-21ksoftirqd/307:13:453
280931499566,8cyclictest23-21ksoftirqd/111:40:081
280931199563,47cyclictest2815538-21cron09:10:000
280931199563,11cyclictest2881289-21kworker/u8:112:30:020
280931199563,11cyclictest2881289-21kworker/u8:112:30:010
2809311995627,3cyclictest131rcu_preempt10:18:040
2809311995626,15cyclictest2912754-21ssh12:38:310
2809311995626,15cyclictest2912754-21ssh12:38:300
280931199561,14cyclictest2897026-21latency_hist12:05:020
2809325995511,9cyclictest37-21ksoftirqd/312:10:413
2809325995511,11cyclictest37-21ksoftirqd/311:34:433
2809325995510,10cyclictest37-21ksoftirqd/309:31:063
2809322995540,7cyclictest2814240-21kworker/2:109:24:562
2809322995512,6cyclictest131rcu_preempt12:35:002
280931499554,8cyclictest23-21ksoftirqd/109:53:251
280931499554,8cyclictest23-21ksoftirqd/109:53:241
2809311995528,12cyclictest2910642-21ssh12:34:120
2809311995527,18cyclictest3467602-21systemd-journal10:15:000
2809311995524,15cyclictest2856103-21ssh10:36:530
280932599545,15cyclictest2814213-21munin-run08:45:003
2809325995432,16cyclictest275-21dbus-daemon08:30:003
2809325995419,10cyclictest2809776-21sh07:20:003
2809325995411,9cyclictest37-21ksoftirqd/311:08:233
2809322995446,5cyclictest2796749-21kworker/2:311:07:312
2809322995445,6cyclictest2796749-21kworker/2:310:26:072
2809322995445,6cyclictest2796749-21kworker/2:309:10:352
2809322995444,7cyclictest2814240-21kworker/2:109:16:512
2809322995439,7cyclictest2889861-21kworker/2:112:12:012
280931499547,7cyclictest23-21ksoftirqd/111:30:351
280931499546,7cyclictest23-21ksoftirqd/111:19:071
280931499545,10cyclictest23-21ksoftirqd/110:22:191
2809314995414,6cyclictest131rcu_preempt10:39:431
2809314995414,5cyclictest131rcu_preempt09:17:241
2809314995412,5cyclictest131rcu_preempt12:37:121
2809314995412,5cyclictest131rcu_preempt12:37:111
280931199545,3cyclictest131rcu_preempt12:05:040
2809311995429,10cyclictest2824129-21ssh09:28:240
2809311995427,12cyclictest2853393-21ssh10:30:580
280932599533,15cyclictest2814476-21apt-config08:45:353
280932599533,14cyclictest2814921-21sed08:55:023
2809325995310,8cyclictest37-21ksoftirqd/311:50:143
2809322995345,5cyclictest2814240-21kworker/2:109:31:572
2809322995344,6cyclictest2796749-21kworker/2:311:27:122
2809322995338,7cyclictest2796749-21kworker/2:310:15:062
280931499537,8cyclictest23-21ksoftirqd/110:02:121
280931499531,5cyclictest131rcu_preempt11:40:021
2809311995345,4cyclictest16850irq/134-800000009:40:010
2809311995325,13cyclictest2818301-21ssh09:15:320
280931199532,3cyclictest131rcu_preempt11:04:290
280932599529,10cyclictest37-21ksoftirqd/309:55:163
2809322995244,5cyclictest2889861-21kworker/2:112:05:212
2809322995244,5cyclictest2889861-21kworker/2:112:03:522
2809322995244,5cyclictest2814240-21kworker/2:110:13:072
2809322995243,6cyclictest2880885-21kworker/2:011:52:292
2809322995243,6cyclictest2814240-21kworker/2:110:23:022
2809322995243,6cyclictest2814240-21kworker/2:109:55:362
2809322995221,11cyclictest30-21ksoftirqd/210:05:012
280931499529,6cyclictest131rcu_preempt09:26:011
280931499529,5cyclictest131rcu_preempt11:49:281
280931499528,5cyclictest131rcu_preempt09:14:311
280931499524,10cyclictest23-21ksoftirqd/110:35:021
280931499522,5cyclictest131rcu_preempt12:19:591
2809314995216,6cyclictest131rcu_preempt10:28:531
2809314995214,5cyclictest131rcu_preempt11:58:191
2809314995210,4cyclictest131rcu_preempt10:49:391
2809314995210,4cyclictest131rcu_preempt10:49:381
280931199523,3cyclictest131rcu_preempt10:42:370
2809311995225,12cyclictest2832306-21ssh09:45:490
2809325995113,8cyclictest37-21ksoftirqd/310:20:033
2809322995143,5cyclictest2899245-21kworker/2:212:35:572
2809322995143,5cyclictest2899245-21kworker/2:212:35:572
2809322995143,5cyclictest2796749-21kworker/2:311:42:172
2809322995142,6cyclictest2855098-21kworker/2:211:03:312
2809322995142,6cyclictest2796749-21kworker/2:310:37:282
2809322995142,6cyclictest2796749-21kworker/2:307:32:572
2809322995141,6cyclictest2880885-21kworker/2:011:35:242
2809322995136,7cyclictest2899245-21kworker/2:212:29:322
2809322995136,7cyclictest2899245-21kworker/2:212:29:312
2809322995128,13cyclictest30-21ksoftirqd/210:55:122
280931499517,7cyclictest23-21ksoftirqd/111:07:361
280931499516,7cyclictest23-21ksoftirqd/110:18:411
280931499516,19cyclictest221rcuc/107:45:031
280931499514,9cyclictest23-21ksoftirqd/112:00:241
2809314995124,10cyclictest23-21ksoftirqd/112:26:331
2809314995124,10cyclictest23-21ksoftirqd/112:26:321
2809314995120,14cyclictest23-21ksoftirqd/109:25:001
2809314995112,11cyclictest23-21ksoftirqd/112:13:361
2809314995111,5cyclictest131rcu_preempt10:06:471
2809314995111,5cyclictest131rcu_preempt10:06:471
2809311995144,5cyclictest19350irq/34-mmc108:35:020
2809311995128,8cyclictest2873045-21ssh11:13:300
280931199512,4cyclictest131rcu_preempt12:20:400
2809311995116,8cyclictest284-21systemd-logind11:23:590
280931199511,28cyclictest0-21swapper/011:38:140
280932599504,9cyclictest37-21ksoftirqd/310:32:053
2809325995030,14cyclictest3467602-21systemd-journal07:45:003
280932299509,5cyclictest131rcu_preempt10:06:432
280932299509,5cyclictest131rcu_preempt10:06:422
280932299508,8cyclictest30-21ksoftirqd/211:35:022
2809322995042,5cyclictest2899245-21kworker/2:212:20:272
2809322995042,5cyclictest2855098-21kworker/2:211:15:202
2809322995042,5cyclictest2855098-21kworker/2:210:52:292
2809322995042,5cyclictest2814240-21kworker/2:109:49:132
2809322995042,5cyclictest2814240-21kworker/2:109:29:372
2809322995042,5cyclictest2796749-21kworker/2:311:10:392
2809322995042,5cyclictest2796749-21kworker/2:310:30:072
2809322995042,5cyclictest2796749-21kworker/2:309:38:392
2809322995041,6cyclictest2814240-21kworker/2:109:52:032
2809322995041,6cyclictest2814240-21kworker/2:109:52:022
2809322995041,6cyclictest2796749-21kworker/2:310:48:492
2809322995041,6cyclictest2796749-21kworker/2:310:48:482
2809322995031,14cyclictest275-21dbus-daemon07:39:592
280931499509,8cyclictest131rcu_preempt09:05:021
280931499507,7cyclictest23-21ksoftirqd/112:24:481
2809314995043,5cyclictest23-21ksoftirqd/108:25:021
2809314995041,6cyclictest221rcuc/110:56:171
280931499504,10cyclictest23-21ksoftirqd/108:40:011
2809314995015,6cyclictest131rcu_preempt11:51:551
2809314995010,4cyclictest131rcu_preempt11:28:481
2809311995042,5cyclictest16850irq/134-800000011:30:020
280931199503,3cyclictest131rcu_preempt12:10:040
2809311995031,13cyclictest3467602-21systemd-journal07:14:590
280932599495,10cyclictest37-21ksoftirqd/308:25:033
2809322994942,4cyclictest2796749-21kworker/2:311:45:302
2809322994941,5cyclictest2855098-21kworker/2:210:43:282
2809322994941,5cyclictest2796749-21kworker/2:311:20:332
2809322994941,5cyclictest2796749-21kworker/2:308:18:282
2809322994940,6cyclictest2889861-21kworker/2:111:57:202
280931499499,5cyclictest131rcu_preempt09:44:191
280931499497,5cyclictest131rcu_preempt09:36:321
280931499497,5cyclictest131rcu_preempt09:33:181
2809314994913,8cyclictest131rcu_preempt11:11:411
2809314994910,10cyclictest131rcu_preempt12:34:301
2809311994929,15cyclictest12-21ksoftirqd/011:05:290
2809311994927,7cyclictest2837006-21ssh09:55:590
280932599481,13cyclictest2893272-21ssh11:57:053
2809322994840,5cyclictest2796749-21kworker/2:308:59:482
2809322994840,5cyclictest2796749-21kworker/2:308:10:122
2809322994823,7cyclictest30-21ksoftirqd/208:10:002
280931499486,7cyclictest23-21ksoftirqd/109:10:011
280931499485,7cyclictest23-21ksoftirqd/109:00:011
280931499483,7cyclictest131rcu_preempt10:41:171
2809314994833,11cyclictest221rcuc/111:02:181
280931199488,27cyclictest19350irq/34-mmc108:05:030
280931199483,4cyclictest131rcu_preempt10:07:200
280931199483,4cyclictest131rcu_preempt10:07:190
2809311994822,3cyclictest131rcu_preempt11:44:310
2809311994818,15cyclictest2893030-21ssh11:56:340
280932599471,13cyclictest2813081-21sed08:20:023
2809322994739,5cyclictest2796749-21kworker/2:309:41:062
2809322994739,5cyclictest2796749-21kworker/2:309:04:492
2809322994739,5cyclictest2796749-21kworker/2:308:41:422
2809322994739,5cyclictest2796749-21kworker/2:307:50:072
2809322994739,5cyclictest2796749-21kworker/2:307:48:372
2809322994739,5cyclictest2796749-21kworker/2:307:48:362
2809322994739,5cyclictest2796749-21kworker/2:307:24:502
2809322994713,6cyclictest131rcu_preempt07:10:452
280931499475,7cyclictest23-21ksoftirqd/107:50:141
2809314994735,9cyclictest221rcuc/109:49:281
280931499472,6cyclictest0-21swapper/108:45:011
2809314994710,6cyclictest131rcu_preempt07:25:001
280931199474,3cyclictest131rcu_preempt09:33:290
2809311994731,9cyclictest275-21dbus-daemon11:35:010
2809311994721,12cyclictest2851587-21ssh10:27:190
2809311994721,11cyclictest2891287-21ssh11:52:470
2809311994720,12cyclictest2835428-21ssh09:52:440
2809311994720,12cyclictest2835428-21ssh09:52:440
2809322994638,5cyclictest2796749-21kworker/2:308:51:572
2809322994638,5cyclictest2796749-21kworker/2:308:49:482
2809322994638,5cyclictest2796749-21kworker/2:308:28:032
2809322994638,5cyclictest2796749-21kworker/2:307:27:222
2809322994637,5cyclictest2796749-21kworker/2:308:01:332
2809322994637,5cyclictest2796749-21kworker/2:308:01:322
280931499467,9cyclictest23-21ksoftirqd/112:07:561
2809314994612,8cyclictest23-21ksoftirqd/108:00:001
2809311994627,12cyclictest275-21dbus-daemon07:20:000
2809311994616,26cyclictest12-21ksoftirqd/008:20:030
2809322994537,5cyclictest2796749-21kworker/2:308:30:072
2809322994537,5cyclictest2796749-21kworker/2:307:40:432
2809322994536,6cyclictest2796749-21kworker/2:307:57:032
280931499459,6cyclictest131rcu_preempt07:10:021
280931499459,6cyclictest131rcu_preempt07:10:011
280931499456,6cyclictest131rcu_preempt08:50:011
2809314994535,6cyclictest23-21ksoftirqd/107:40:001
2809314994532,10cyclictest275-21dbus-daemon08:55:011
2809314994510,31cyclictest1-21systemd07:15:011
280931199452,7cyclictest275-21dbus-daemon08:15:000
2809311994527,14cyclictest2863651-21ssh10:53:180
2809311994510,31cyclictest2811999-21latency_hist08:00:020
2809322994437,4cyclictest2796749-21kworker/2:308:23:132
2809322994436,5cyclictest2796749-21kworker/2:309:05:582
2809322994436,5cyclictest2796749-21kworker/2:308:36:542
280931499449,5cyclictest131rcu_preempt08:34:481
2809314994412,5cyclictest131rcu_preempt10:12:311
2809311994430,10cyclictest3467602-21systemd-journal08:36:470
2809311994426,14cyclictest2814730-21sshd08:50:470
280931199442,13cyclictest2815038-21tr09:00:010
280931499437,5cyclictest131rcu_preempt08:20:011
280931499431,7cyclictest0-21swapper/108:29:321
280931199431,32cyclictest2810274-21kworker/u8:007:50:020
280931199431,32cyclictest2810274-21kworker/u8:007:50:010
2809322994134,5cyclictest2796749-21kworker/2:307:15:032
2809325994029,6cyclictest2812378-21munin-plugin-st08:10:003
2809325994028,9cyclictest2811825-21sshd07:58:453
280931499408,5cyclictest131rcu_preempt08:04:271
280931499408,5cyclictest131rcu_preempt08:04:261
2809314994029,9cyclictest23-21ksoftirqd/108:10:011
2809311994033,4cyclictest16950irq/224-800000007:35:010
2809311994025,11cyclictest309-21in:imuxsock07:40:010
2809325993927,9cyclictest37-21ksoftirqd/308:35:013
2809322993914,6cyclictest131rcu_preempt07:10:032
2809322993914,6cyclictest131rcu_preempt07:10:022
280931499398,5cyclictest131rcu_preempt08:11:471
2809314993913,5cyclictest131rcu_preempt07:50:021
2809314993913,5cyclictest131rcu_preempt07:50:011
280931499391,10cyclictest150-21pr/ttyS207:25:211
2809314993910,12cyclictest23-21ksoftirqd/107:30:451
2809311993932,4cyclictest16950irq/224-800000007:30:000
280931499381,9cyclictest0-21swapper/107:18:301
2809311993819,15cyclictest2812103-21sshd08:02:460
2809311993819,15cyclictest2812103-21sshd08:02:450
2809325993722,11cyclictest1-21systemd08:15:013
280931199376,3cyclictest131rcu_preempt07:10:020
280931199376,3cyclictest131rcu_preempt07:10:010
2809311993724,9cyclictest12-21ksoftirqd/009:05:020
2809311993722,11cyclictest364-21snmpd07:23:330
2809325993621,10cyclictest309-21in:imuxsock07:25:033
280931199362,30cyclictest0-21swapper/007:53:000
2809311993622,11cyclictest2813156-21sshd08:23:470
280931199361,11cyclictest364-21snmpd07:40:190
280932599359,17cyclictest2809307-21cyclictest07:50:023
280932599359,17cyclictest2809307-21cyclictest07:50:013
280931199351,31cyclictest0-21swapper/008:40:290
280931199341,29cyclictest0-21swapper/008:26:570
280932599335,7cyclictest37-21ksoftirqd/307:31:043
280932599321,7cyclictest0-21swapper/308:01:333
280932599321,7cyclictest0-21swapper/308:01:323
2809325993011,16cyclictest2809343-21latency_hist07:10:013
2809325993011,16cyclictest2809343-21latency_hist07:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional