You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-05 - 16:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot3s.osadl.org (updated Wed Aug 05, 2026 12:45:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
217424499931,17cyclictest131rcu_preempt12:35:033
2174230997412,40cyclictest275-21dbus-daemon11:15:010
2174230996914,32cyclictest2192086-21ssh09:35:020
217423099691,2cyclictest0-21swapper/008:45:020
2174240996818,24cyclictest30-21ksoftirqd/210:40:032
2174230996724,38cyclictest2252599-21cat11:40:000
2174230996717,34cyclictest12-21ksoftirqd/009:54:060
2174230996516,7cyclictest131rcu_preempt12:35:010
217423099645,6cyclictest131rcu_preempt12:15:000
217423099645,5cyclictest131rcu_preempt09:45:030
2174230996324,34cyclictest12-21ksoftirqd/010:50:220
2174230996313,6cyclictest131rcu_preempt11:05:020
2174230996232,14cyclictest2275161-21ssh12:26:280
2174230996227,31cyclictest12-21ksoftirqd/010:33:000
2174230996118,26cyclictest12-21ksoftirqd/010:37:270
2174230995929,9cyclictest12-21ksoftirqd/010:10:010
2174230995928,15cyclictest2253173-21ssh11:40:440
2174230995913,31cyclictest12-21ksoftirqd/009:23:490
217423099587,29cyclictest12-21ksoftirqd/012:05:020
2174230995821,33cyclictest12-21ksoftirqd/010:23:080
2174230995814,8cyclictest12-21ksoftirqd/011:00:000
2174230995753,2cyclictest0-21swapper/008:15:020
217423099574,7cyclictest131rcu_preempt11:25:010
2174230995739,9cyclictest309-21in:imuxsock08:20:000
217423099572,5cyclictest131rcu_preempt09:10:520
2174230995724,17cyclictest2204980-21ssh10:02:170
2174230995721,4cyclictest12-21ksoftirqd/011:07:130
2174230995721,4cyclictest12-21ksoftirqd/011:07:120
2174230995628,12cyclictest2224104-21ssh10:42:240
2174230995619,33cyclictest12-21ksoftirqd/009:20:010
2174230995619,33cyclictest12-21ksoftirqd/009:20:000
2174230995527,12cyclictest2188847-21ssh09:28:290
2174230995527,12cyclictest2188847-21ssh09:28:280
2174230995516,12cyclictest2245857-21ssh11:26:220
2174230995434,16cyclictest2193448-21sshd09:38:150
2174230995434,16cyclictest2193448-21sshd09:38:150
2174230995427,5cyclictest131rcu_preempt12:05:030
2174230995427,12cyclictest2256296-21ssh11:47:320
2174230995427,12cyclictest2256296-21ssh11:47:310
2174230995426,7cyclictest12-21ksoftirqd/012:20:030
2174244995336,12cyclictest2246030-21sshd11:26:293
2174235995313,6cyclictest131rcu_preempt11:10:021
2174235995313,6cyclictest131rcu_preempt11:10:021
2174235995234,13cyclictest1-21systemd07:25:011
2174230995233,13cyclictest3467602-21systemd-journal09:05:020
2174230995232,14cyclictest275-21dbus-daemon09:10:000
2174230995230,16cyclictest3467602-21systemd-journal09:59:180
2174230995216,32cyclictest2175650-21sendmail07:39:590
2174230995215,31cyclictest131rcu_preempt11:58:560
2174230995215,31cyclictest131rcu_preempt11:58:550
2174230995214,7cyclictest274-21cron09:50:000
217424099518,30cyclictest291rcuc/211:35:012
2174240995143,5cyclictest2244301-21kworker/2:111:44:252
2174240995143,5cyclictest2173662-21kworker/2:209:05:032
2174240995142,5cyclictest2173662-21kworker/2:210:03:152
2174230995127,17cyclictest275-21dbus-daemon08:50:020
2174230995127,17cyclictest275-21dbus-daemon08:50:010
2174230995125,11cyclictest2280795-21ssh12:38:220
2174230995115,12cyclictest12-21ksoftirqd/008:55:010
2174240995042,5cyclictest2244301-21kworker/2:111:46:362
2174240995042,5cyclictest2244301-21kworker/2:111:46:352
2174240995042,5cyclictest2244301-21kworker/2:111:35:072
2174240995042,5cyclictest2177647-21kworker/2:109:17:142
2174240995042,5cyclictest2177647-21kworker/2:109:17:142
2174240995042,5cyclictest2173662-21kworker/2:211:25:572
2174240995042,5cyclictest2173662-21kworker/2:210:31:132
2174240995041,6cyclictest2261325-21kworker/2:012:30:012
2174240995041,6cyclictest2177647-21kworker/2:109:12:382
2174240995041,6cyclictest2173662-21kworker/2:212:38:282
2174240995035,11cyclictest3467602-21systemd-journal11:55:032
2174240995035,11cyclictest3467602-21systemd-journal11:55:022
217424099503,17cyclictest131rcu_preempt08:30:002
2174244994934,11cyclictest3467602-21systemd-journal07:45:003
2174244994933,11cyclictest2272124-21sed12:20:033
2174240994942,4cyclictest2173662-21kworker/2:209:55:402
2174240994941,5cyclictest2237217-21kworker/2:311:16:372
2174240994941,5cyclictest2177647-21kworker/2:109:32:462
2174240994941,5cyclictest2173662-21kworker/2:212:07:012
2174240994941,5cyclictest2173662-21kworker/2:211:51:302
2174240994941,5cyclictest2173662-21kworker/2:210:05:122
2174240994941,5cyclictest2173662-21kworker/2:209:28:032
2174240994941,5cyclictest2173662-21kworker/2:209:28:032
2174240994940,5cyclictest2173662-21kworker/2:212:13:162
2174240994936,10cyclictest2201064-21sshd09:54:172
2174235994932,13cyclictest275-21dbus-daemon09:20:011
2174235994932,13cyclictest275-21dbus-daemon09:20:001
217423099492,9cyclictest2174661-21sendmail07:20:000
217423099492,6cyclictest131rcu_preempt07:10:020
2174230994924,22cyclictest2258628-21ssh11:52:220
2174230994918,10cyclictest12-21ksoftirqd/010:15:010
2174230994915,30cyclictest0-21swapper/010:16:520
2174244994835,9cyclictest1-21systemd08:20:013
2174244994833,12cyclictest3467602-21systemd-journal08:33:093
217424499481,4cyclictest131rcu_preempt11:09:023
217424499481,4cyclictest131rcu_preempt11:09:013
2174240994840,5cyclictest2237217-21kworker/2:311:14:212
2174240994840,5cyclictest2177647-21kworker/2:109:22:322
2174235994834,10cyclictest3467602-21systemd-journal12:07:471
2174235994831,11cyclictest3467602-21systemd-journal09:55:021
217423099489,15cyclictest12-21ksoftirqd/010:26:500
2174230994836,9cyclictest16950irq/224-800000008:40:000
2174230994834,9cyclictest12-21ksoftirqd/007:50:010
2174230994818,7cyclictest12-21ksoftirqd/010:48:370
2174244994734,10cyclictest2255353-21sshd11:45:173
2174244994734,10cyclictest2255353-21sshd11:45:163
2174244994733,10cyclictest2273128-21sshd12:22:173
2174244994732,10cyclictest37-21ksoftirqd/310:25:013
2174240994740,4cyclictest2261325-21kworker/2:012:01:472
2174240994739,5cyclictest2237217-21kworker/2:311:21:292
2174240994739,5cyclictest2173662-21kworker/2:210:46:212
2174240994739,5cyclictest2173662-21kworker/2:209:43:472
2174240994738,5cyclictest2174951-21kworker/2:007:55:002
217423599474,4cyclictest131rcu_preempt11:25:021
2174235994732,10cyclictest1-21systemd07:45:001
217423599472,3cyclictest131rcu_preempt10:04:281
217423099479,6cyclictest131rcu_preempt07:45:020
217423099472,39cyclictest309-21in:imuxsock07:25:020
217423099472,25cyclictest12-21ksoftirqd/007:30:000
2174230994713,10cyclictest12-21ksoftirqd/011:31:070
2174230994712,6cyclictest131rcu_preempt12:24:590
2174240994639,4cyclictest2173662-21kworker/2:210:40:182
2174240994638,5cyclictest2222951-21kworker/2:010:52:212
2174240994638,5cyclictest2173662-21kworker/2:212:22:212
2174240994635,9cyclictest0-21swapper/207:10:012
217423599467,9cyclictest23-21ksoftirqd/109:00:001
2174235994633,8cyclictest1-21systemd11:26:241
2174230994620,8cyclictest12-21ksoftirqd/007:55:010
2174230994614,6cyclictest131rcu_preempt11:20:000
217423099461,41cyclictest1-21systemd08:10:000
217423099461,36cyclictest2173632-21kworker/u8:007:10:030
217423099461,36cyclictest2173632-21kworker/u8:007:10:020
2174230994612,6cyclictest131rcu_preempt08:25:010
2174240994537,5cyclictest2222951-21kworker/2:010:55:172
2174235994528,12cyclictest275-21dbus-daemon12:30:011
217423099452,38cyclictest2179578-21systemd-run09:00:000
217424499443,3cyclictest131rcu_preempt08:00:003
2174244994430,10cyclictest309-21in:imuxsock11:40:013
2174244994413,6cyclictest131rcu_preempt09:40:013
2174244994413,6cyclictest131rcu_preempt09:40:003
2174240994436,5cyclictest2222951-21kworker/2:011:05:532
2174240994436,5cyclictest2222951-21kworker/2:011:05:532
2174240994436,5cyclictest2173662-21kworker/2:209:45:062
217423599443,3cyclictest131rcu_preempt12:37:521
217423599443,3cyclictest131rcu_preempt12:20:131
2174235994428,11cyclictest275-21dbus-daemon09:40:011
2174235994428,11cyclictest275-21dbus-daemon09:40:011
217423599441,3cyclictest131rcu_preempt07:50:021
2174230994437,4cyclictest16950irq/224-800000008:00:010
2174230994411,6cyclictest131rcu_preempt07:35:010
2174230994411,6cyclictest131rcu_preempt07:35:000
2174244994331,8cyclictest2212938-21sshd10:19:163
2174244994328,11cyclictest275-21dbus-daemon10:15:013
2174244994327,12cyclictest275-21dbus-daemon10:35:003
2174244994326,13cyclictest275-21dbus-daemon10:45:013
2174240994335,5cyclictest2173662-21kworker/2:209:36:442
2174240994335,5cyclictest2173662-21kworker/2:209:36:432
2174240994329,10cyclictest275-21dbus-daemon07:25:022
217423599433,4cyclictest131rcu_preempt10:39:021
217423599433,3cyclictest131rcu_preempt09:31:191
2174235994329,10cyclictest2270284-21systemd12:16:231
217423599432,3cyclictest131rcu_preempt11:35:021
2174235994323,15cyclictest3467602-21systemd-journal12:00:011
2174235994323,15cyclictest3467602-21systemd-journal12:00:011
2174235994315,6cyclictest23-21ksoftirqd/111:17:341
217423599431,3cyclictest131rcu_preempt10:44:471
2174244994231,8cyclictest37-21ksoftirqd/307:35:003
2174244994231,8cyclictest37-21ksoftirqd/307:35:003
2174244994231,8cyclictest2261360-21sshd11:58:103
2174244994231,8cyclictest2261360-21sshd11:58:093
2174244994229,9cyclictest3467602-21systemd-journal07:55:003
2174244994227,11cyclictest275-21dbus-daemon09:20:023
2174244994227,11cyclictest275-21dbus-daemon09:20:013
2174244994214,14cyclictest37-21ksoftirqd/312:35:383
2174240994234,5cyclictest2173662-21kworker/2:208:45:012
2174240994234,5cyclictest2173662-21kworker/2:208:13:172
2174240994222,15cyclictest275-21dbus-daemon12:20:012
2174240994213,7cyclictest2222951-21kworker/2:011:01:252
2174235994230,10cyclictest23-21ksoftirqd/110:16:261
2174235994229,9cyclictest2193657-1kworker/u9:009:45:021
2174235994227,12cyclictest23-21ksoftirqd/111:45:001
2174235994227,11cyclictest1-21systemd09:25:001
2174235994227,10cyclictest309-21in:imuxsock11:38:181
217423599422,3cyclictest131rcu_preempt12:11:191
2174230994214,19cyclictest275-21dbus-daemon08:05:010
2174244994134,4cyclictest2232344-21munin-plugin-st11:00:013
217424499413,3cyclictest131rcu_preempt10:00:143
2174244994131,6cyclictest2222843-21cron10:40:013
2174244994129,8cyclictest3467602-21systemd-journal09:25:033
2174244994129,8cyclictest3467602-21systemd-journal09:25:023
2174244994129,8cyclictest275-21dbus-daemon09:05:013
2174244994128,10cyclictest2267085-21cron12:10:013
2174244994127,11cyclictest37-21ksoftirqd/311:55:023
2174244994116,3cyclictest37-21ksoftirqd/312:12:243
2174244994116,12cyclictest1-21systemd09:25:003
217424499411,3cyclictest131rcu_preempt11:10:413
2174240994136,3cyclictest2261325-21kworker/2:012:30:052
2174240994133,5cyclictest2177647-21kworker/2:108:36:572
2174240994133,5cyclictest2173662-21kworker/2:209:08:282
217424099413,10cyclictest131rcu_preempt07:30:012
2174240994130,7cyclictest2177844-21sshd08:23:082
2174240994128,10cyclictest30-21ksoftirqd/210:15:112
217423599414,3cyclictest131rcu_preempt09:28:151
217423599414,3cyclictest131rcu_preempt09:28:141
217423599413,3cyclictest131rcu_preempt07:15:021
217423599413,3cyclictest131rcu_preempt07:15:011
2174235994129,8cyclictest23-21ksoftirqd/108:00:001
2174235994128,9cyclictest309-21in:imuxsock11:03:311
2174235994128,9cyclictest1-21systemd08:50:021
2174235994128,9cyclictest1-21systemd08:50:011
2174235994127,10cyclictest309-21in:imuxsock10:30:011
2174235994124,4cyclictest23-21ksoftirqd/112:05:001
217423599412,3cyclictest131rcu_preempt10:50:471
217423599411,9cyclictest3467602-21systemd-journal07:35:021
217423599411,9cyclictest3467602-21systemd-journal07:35:021
2174230994126,11cyclictest275-21dbus-daemon08:25:490
217424499405,9cyclictest37-21ksoftirqd/307:50:033
2174244994027,9cyclictest37-21ksoftirqd/310:07:193
2174244994027,11cyclictest37-21ksoftirqd/310:47:273
2174244994026,10cyclictest37-21ksoftirqd/309:10:493
2174244994016,10cyclictest37-21ksoftirqd/311:25:023
2174240994028,9cyclictest30-21ksoftirqd/210:10:552
2174240994026,9cyclictest275-21dbus-daemon07:20:012
2174240994025,13cyclictest30-21ksoftirqd/208:05:022
217424099401,23cyclictest2175620-21kworker/u8:007:55:032
2174235994026,11cyclictest3467602-21systemd-journal10:48:311
2174235994026,10cyclictest23-21ksoftirqd/108:55:011
2174235994016,6cyclictest131rcu_preempt09:55:001
217423599401,3cyclictest131rcu_preempt09:13:101
217424499394,3cyclictest131rcu_preempt11:01:203
2174244993930,7cyclictest37-21ksoftirqd/312:26:393
2174244993925,10cyclictest275-21dbus-daemon08:10:003
2174244993923,12cyclictest275-21dbus-daemon09:45:003
2174244993923,11cyclictest2174666-21run-parts07:20:003
2174240993931,5cyclictest2173662-21kworker/2:210:27:162
2174240993926,9cyclictest3467602-21systemd-journal07:50:002
2174235993924,11cyclictest3467602-21systemd-journal08:20:021
2174235993924,11cyclictest1-21systemd08:05:011
2174235993922,13cyclictest275-21dbus-daemon11:55:001
2174235993920,5cyclictest131rcu_preempt09:50:021
2174235993915,7cyclictest131rcu_preempt10:06:451
217423599391,3cyclictest131rcu_preempt09:10:011
217423599391,3cyclictest131rcu_preempt07:10:021
217424499387,6cyclictest0-21swapper/311:17:563
217424499385,3cyclictest131rcu_preempt09:58:353
2174244993829,6cyclictest37-21ksoftirqd/310:29:073
2174244993827,7cyclictest2228785-21sshd10:52:173
217424499382,5cyclictest2179535-21modprobe09:00:003
2174244993825,10cyclictest2177535-21sed08:15:023
2174244993824,10cyclictest37-21ksoftirqd/309:52:363
2174244993823,11cyclictest37-21ksoftirqd/311:41:343
217424499381,3cyclictest131rcu_preempt12:05:023
217424499381,3cyclictest131rcu_preempt09:50:023
2174240993829,5cyclictest2161692-21kworker/2:107:14:392
2174240993829,5cyclictest2161692-21kworker/2:107:14:382
217424099381,14cyclictest2214412-21ssh10:21:592
2174235993828,8cyclictest23-21ksoftirqd/112:32:581
2174235993827,9cyclictest23-21ksoftirqd/111:14:591
2174235993824,10cyclictest3467602-21systemd-journal07:27:321
2174235993823,11cyclictest3467602-21systemd-journal10:35:001
217423099387,5cyclictest131rcu_preempt08:35:000
217424499372,4cyclictest131rcu_preempt09:10:033
2174244993723,11cyclictest37-21ksoftirqd/311:34:333
217424499371,4cyclictest131rcu_preempt08:50:023
217424499371,4cyclictest131rcu_preempt08:50:023
2174240993725,8cyclictest275-21dbus-daemon07:35:012
2174240993725,8cyclictest275-21dbus-daemon07:35:002
2174240993722,10cyclictest2175642-21run-parts07:39:592
2174235993726,8cyclictest23-21ksoftirqd/110:20:241
2174235993723,10cyclictest23-21ksoftirqd/108:35:011
2174235993716,6cyclictest131rcu_preempt10:15:001
2174244993626,7cyclictest37-21ksoftirqd/308:40:023
2174244993625,9cyclictest37-21ksoftirqd/307:30:013
2174244993624,9cyclictest2177996-21wc08:25:023
2174244993623,9cyclictest2174440-21sort07:15:023
2174244993623,9cyclictest2174440-21sort07:15:013
2174244993616,14cyclictest275-21dbus-daemon08:45:013
217424499361,19cyclictest0-21swapper/309:32:003
217423599363,6cyclictest23-21ksoftirqd/111:45:531
217423599363,6cyclictest23-21ksoftirqd/111:45:531
2174235993626,6cyclictest309-21in:imuxsock08:45:001
2174235993624,7cyclictest23-21ksoftirqd/108:40:001
217423599361,3cyclictest131rcu_preempt10:59:341
2174235993613,6cyclictest131rcu_preempt08:20:021
2174244993519,12cyclictest2175652-21tr07:40:003
2174240993527,5cyclictest2173662-21kworker/2:208:53:002
217423599352,3cyclictest131rcu_preempt07:40:021
2174235993521,10cyclictest23-21ksoftirqd/108:07:011
2174235993518,14cyclictest3467602-21systemd-journal07:50:021
217424499341,9cyclictest2176883-21systemd-run08:05:023
2174240993423,9cyclictest30-21ksoftirqd/208:49:362
2174240993423,9cyclictest30-21ksoftirqd/208:49:352
217424099341,3cyclictest131rcu_preempt08:20:002
2174235993421,9cyclictest3467602-21systemd-journal08:15:001
2174244993325,6cyclictest37-21ksoftirqd/307:20:023
2174244993321,10cyclictest37-21ksoftirqd/308:55:013
2174240993325,4cyclictest2179628-21latency_hist09:00:012
217423599334,3cyclictest131rcu_preempt07:20:001
2174235993323,7cyclictest23-21ksoftirqd/108:30:011
2174244993219,10cyclictest37-21ksoftirqd/308:28:213
217424099323,5cyclictest2177252-21wc08:10:012
2174240993223,7cyclictest30-21ksoftirqd/208:35:012
2174240993222,6cyclictest275-21dbus-daemon07:45:012
217424499311,3cyclictest131rcu_preempt07:10:023
2174235993020,6cyclictest2180020-21latency_hist09:05:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional