You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-28 - 20:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Fri Aug 28, 2026 12:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2694912991437,61cyclictest2705907-21ssh09:20:020
269491299112105,5cyclictest12-21ksoftirqd/012:35:020
2694912997217,4cyclictest12-21ksoftirqd/011:45:000
2694912997217,4cyclictest12-21ksoftirqd/011:44:590
2694912996818,16cyclictest131rcu_preempt10:30:400
2694912996615,21cyclictest2731648-21kworker/u8:210:15:010
2694919996541,8cyclictest2748055-21kworker/2:212:16:142
2694912996319,39cyclictest16850irq/134-800000012:10:010
2694919996234,7cyclictest2748055-21kworker/2:211:12:042
2694919996139,9cyclictest2686948-21kworker/2:109:14:242
2694922995944,11cyclictest1-21systemd12:05:013
2694912995928,15cyclictest2729193-21ssh10:09:470
2694912995928,15cyclictest2703216-21ssh09:14:380
2694912995915,39cyclictest16850irq/134-800000011:30:020
2694912995839,13cyclictest3467602-21systemd-journal12:00:000
2694914995749,5cyclictest23-21ksoftirqd/108:00:021
2694914995733,19cyclictest275-21dbus-daemon09:45:011
269491299573,48cyclictest2700834-21unin-run09:05:010
2694912995729,12cyclictest2763971-21ssh11:23:400
2694912995727,16cyclictest2723475-21ssh09:57:280
269491299571,39cyclictest131rcu_preempt09:39:390
2694914995633,18cyclictest275-21dbus-daemon08:45:011
2694912995630,21cyclictest2748014-21ssh10:50:000
2694912995628,13cyclictest2708569-21ssh09:25:350
2694912995617,34cyclictest16850irq/134-800000012:12:490
269491299561,39cyclictest2719420-21ssh09:49:090
2694919995545,6cyclictest2686948-21kworker/2:109:17:182
2694922995424,19cyclictest361rcuc/310:45:023
2694919995446,5cyclictest2748055-21kworker/2:212:24:292
2694912995435,15cyclictest1-21systemd08:50:010
2694912995427,12cyclictest2774886-21ssh11:46:330
2694912995425,21cyclictest2701073-21diskmemload10:57:350
2694912995421,16cyclictest2750423-21ssh10:55:010
2694912995421,16cyclictest2750423-21ssh10:55:010
2694912995416,19cyclictest16850irq/134-800000012:29:280
269491299541,47cyclictest27243621kworker/u8:010:00:440
2694912995414,20cyclictest16850irq/134-800000009:44:300
269491299541,38cyclictest2799609-21ssh12:38:590
2694912995413,33cyclictest16850irq/134-800000012:05:010
2694912995413,21cyclictest16850irq/134-800000011:17:450
2694919995345,5cyclictest2700301-21kworker/2:309:59:572
2694919995345,5cyclictest2686948-21kworker/2:110:52:382
2694919995345,5cyclictest2686948-21kworker/2:110:52:372
2694919995345,5cyclictest2686948-21kworker/2:110:41:392
2694919995344,6cyclictest2748055-21kworker/2:211:48:362
2694914995334,14cyclictest275-21dbus-daemon12:40:001
2694914995334,14cyclictest275-21dbus-daemon09:05:001
2694912995327,11cyclictest2789242-21ssh12:16:520
2694912995316,33cyclictest16850irq/134-800000011:11:270
2694912995315,35cyclictest16850irq/134-800000011:34:300
2694912995314,34cyclictest16850irq/134-800000011:02:200
269491299531,36cyclictest2720835-21ssh09:51:450
2694912995313,36cyclictest16850irq/134-800000009:33:360
2694912995313,20cyclictest16850irq/134-800000011:09:360
2694922995235,13cyclictest275-21dbus-daemon07:15:013
2694919995244,5cyclictest2748055-21kworker/2:212:01:132
2694919995244,5cyclictest2686948-21kworker/2:109:30:412
2694919995243,6cyclictest2700301-21kworker/2:310:15:092
2694919995243,6cyclictest2700301-21kworker/2:310:15:092
2694919995243,5cyclictest2748055-21kworker/2:212:26:152
2694919995243,5cyclictest2748055-21kworker/2:212:08:142
2694919995243,5cyclictest2748055-21kworker/2:211:06:352
2694919995242,6cyclictest2686948-21kworker/2:109:25:022
269491999523,6cyclictest131rcu_preempt09:48:112
269491299524,3cyclictest131rcu_preempt10:40:030
2694912995230,15cyclictest0-21swapper/010:15:160
2694912995230,15cyclictest0-21swapper/010:15:150
2694912995216,32cyclictest16850irq/134-800000010:20:290
2694912995214,34cyclictest16850irq/134-800000010:25:180
2694912995214,34cyclictest16850irq/134-800000009:22:460
2694912995213,34cyclictest16850irq/134-800000012:25:000
269492299515,3cyclictest131rcu_preempt11:14:133
2694919995143,5cyclictest2748055-21kworker/2:211:44:342
2694919995143,5cyclictest2748055-21kworker/2:211:44:342
2694919995143,5cyclictest2748055-21kworker/2:211:05:022
2694919995143,5cyclictest2700301-21kworker/2:310:31:432
2694919995143,5cyclictest2686948-21kworker/2:110:37:102
2694919995143,5cyclictest2686948-21kworker/2:107:35:012
2694919995142,5cyclictest2686948-21kworker/2:108:25:002
2694919995134,13cyclictest1-21systemd08:10:012
2694919995134,13cyclictest1-21systemd08:10:012
2694919995123,16cyclictest30-21ksoftirqd/211:17:512
2694912995122,13cyclictest2743981-21ssh10:41:030
2694912995112,19cyclictest12-21ksoftirqd/011:51:370
2694922995036,10cyclictest275-21dbus-daemon10:30:013
2694922995029,9cyclictest37-21ksoftirqd/310:00:003
2694919995042,5cyclictest2788215-21kworker/2:312:35:422
2694919995042,5cyclictest2748055-21kworker/2:212:00:012
2694919995042,5cyclictest2686948-21kworker/2:110:49:242
2694919995042,5cyclictest2686948-21kworker/2:109:44:482
2694919995042,5cyclictest2686948-21kworker/2:109:28:572
2694919995041,5cyclictest2700301-21kworker/2:310:26:262
2694919995041,5cyclictest2700301-21kworker/2:310:10:052
2694919995040,6cyclictest2700301-21kworker/2:310:08:362
2694919995029,7cyclictest2788215-21kworker/2:312:34:312
2694919995015,8cyclictest30-21ksoftirqd/211:25:022
2694914995028,16cyclictest275-21dbus-daemon09:25:011
2694914995028,11cyclictest2702731-21ssh09:13:261
2694912995035,11cyclictest3467602-21systemd-journal08:45:010
269491999498,5cyclictest131rcu_preempt08:30:012
2694919994941,5cyclictest2748055-21kworker/2:211:32:282
2694919994941,5cyclictest2748055-21kworker/2:210:55:302
2694919994940,7cyclictest30-21ksoftirqd/208:55:022
2694919994940,6cyclictest2686948-21kworker/2:108:40:022
2694919994940,5cyclictest2748055-21kworker/2:212:14:442
2694914994931,14cyclictest1-21systemd10:00:011
2694914994928,16cyclictest275-21dbus-daemon12:20:011
2694914994921,19cyclictest23-21ksoftirqd/112:04:281
269491299492,26cyclictest131rcu_preempt08:34:000
269491299492,26cyclictest131rcu_preempt08:34:000
2694912994915,30cyclictest16850irq/134-800000011:36:320
2694912994915,30cyclictest16850irq/134-800000011:36:320
269492299481,20cyclictest2778559-21ssh11:54:363
2694919994841,4cyclictest2748055-21kworker/2:211:51:142
2694919994840,5cyclictest2748055-21kworker/2:211:26:282
2694919994840,5cyclictest2700301-21kworker/2:310:21:582
2694919994840,5cyclictest2686948-21kworker/2:109:36:402
2694919994839,5cyclictest2686948-21kworker/2:108:00:002
2694919994838,7cyclictest2686948-21kworker/2:108:44:292
2694919994739,5cyclictest2748055-21kworker/2:211:37:062
2694919994739,5cyclictest2748055-21kworker/2:211:37:062
2694919994739,5cyclictest2686948-21kworker/2:108:50:012
2694919994739,5cyclictest2686948-21kworker/2:108:15:012
2694919994737,5cyclictest2686948-21kworker/2:107:50:022
2694919994726,7cyclictest2700301-21kworker/2:309:51:422
269491499478,5cyclictest131rcu_preempt10:05:001
2694914994732,11cyclictest1-21systemd10:25:001
2694914994729,14cyclictest2699506-21cron08:40:001
269492299463,4cyclictest131rcu_preempt09:16:383
2694919994630,10cyclictest2700544-21cron09:00:002
2694919994630,10cyclictest2700544-21cron09:00:002
269491499466,5cyclictest131rcu_preempt10:50:431
269491499466,5cyclictest131rcu_preempt10:50:421
269491499465,5cyclictest131rcu_preempt11:49:111
2694914994630,8cyclictest309-21in:imuxsock08:35:011
2694914994630,8cyclictest309-21in:imuxsock08:35:011
2694914994629,13cyclictest23-21ksoftirqd/111:00:021
2694914994627,13cyclictest275-21dbus-daemon07:25:001
2694914994613,5cyclictest131rcu_preempt08:15:001
269491299468,6cyclictest131rcu_preempt07:20:020
2694912994638,4cyclictest16950irq/224-800000008:00:010
2694919994537,5cyclictest2700301-21kworker/2:310:04:132
2694919994537,5cyclictest2686948-21kworker/2:107:25:022
2694919994537,5cyclictest2686948-21kworker/2:107:17:182
269491499456,10cyclictest23-21ksoftirqd/110:50:021
269491499455,8cyclictest23-21ksoftirqd/110:15:031
2694914994530,8cyclictest309-21in:imuxsock11:30:001
2694914994530,12cyclictest23-21ksoftirqd/112:31:231
2694914994525,14cyclictest275-21dbus-daemon07:45:011
2694914994524,16cyclictest2705871-21grep09:20:021
2694912994537,5cyclictest16950irq/224-800000008:20:010
269492299442,4cyclictest131rcu_preempt11:24:153
269492299441,3cyclictest131rcu_preempt12:08:433
269492299441,3cyclictest131rcu_preempt11:35:163
269492299441,3cyclictest131rcu_preempt11:35:163
269492299441,3cyclictest131rcu_preempt09:26:533
2694919994436,5cyclictest2686948-21kworker/2:108:05:002
2694919994435,5cyclictest2686948-21kworker/2:109:05:012
2694914994434,6cyclictest2729310-21munin-plugin-st10:10:001
269491499441,11cyclictest2762338-21latency_hist11:20:031
269492299433,9cyclictest37-21ksoftirqd/310:11:453
2694922994329,12cyclictest37-21ksoftirqd/312:12:203
2694922994328,11cyclictest275-21dbus-daemon10:05:003
269492299432,4cyclictest131rcu_preempt11:43:443
269492299432,4cyclictest131rcu_preempt11:43:443
269492299431,7cyclictest0-21swapper/312:19:363
269492299431,3cyclictest131rcu_preempt09:14:523
269491499437,28cyclictest2701064-21modprobe09:10:001
269491499437,27cyclictest2700552-21grep09:00:001
269491499437,27cyclictest2700552-21grep09:00:001
269491499436,6cyclictest131rcu_preempt11:00:001
269491499436,5cyclictest131rcu_preempt11:56:041
269491499436,5cyclictest131rcu_preempt10:20:031
269491499436,5cyclictest131rcu_preempt10:20:021
2694914994332,8cyclictest23-21ksoftirqd/107:30:001
2694914994329,10cyclictest2715121-21sendmail09:40:001
2694914994313,5cyclictest131rcu_preempt11:10:021
2694914994311,7cyclictest131rcu_preempt07:15:011
2694914994311,6cyclictest131rcu_preempt11:55:001
2694912994337,3cyclictest16950irq/224-800000007:50:010
2694912994332,8cyclictest1-21systemd09:10:010
2694922994231,7cyclictest37-21ksoftirqd/309:50:003
2694922994230,9cyclictest37-21ksoftirqd/311:20:013
2694922994229,9cyclictest275-21dbus-daemon10:55:013
2694922994229,9cyclictest275-21dbus-daemon10:55:013
2694922994228,11cyclictest275-21dbus-daemon08:55:013
2694922994227,11cyclictest3467602-21systemd-journal12:40:013
269492299422,3cyclictest131rcu_preempt11:48:393
269492299422,3cyclictest131rcu_preempt11:27:053
269491499427,26cyclictest2738653-21modprobe10:30:001
269491499426,8cyclictest23-21ksoftirqd/109:50:011
269491499425,5cyclictest131rcu_preempt12:24:041
269491499424,6cyclictest131rcu_preempt12:10:001
2694914994229,10cyclictest23-21ksoftirqd/109:32:381
2694914994227,11cyclictest1-21systemd09:30:001
2694914994220,4cyclictest23-21ksoftirqd/111:36:221
2694914994220,4cyclictest23-21ksoftirqd/111:36:221
2694914994211,6cyclictest131rcu_preempt07:50:011
2694914994210,5cyclictest131rcu_preempt11:22:271
269491299428,3cyclictest131rcu_preempt08:10:000
269491299428,3cyclictest131rcu_preempt08:10:000
269491299424,3cyclictest131rcu_preempt07:35:010
2694912994230,7cyclictest2700529-21cron09:00:000
2694912994230,7cyclictest2700529-21cron09:00:000
269491299422,36cyclictest2699650-21sed08:40:010
2694922994122,10cyclictest37-21ksoftirqd/307:25:013
269492299411,6cyclictest0-21swapper/309:50:273
269492299411,3cyclictest131rcu_preempt10:06:093
2694919994124,12cyclictest275-21dbus-daemon09:10:012
269491999412,3cyclictest131rcu_preempt08:17:002
2694919994121,6cyclictest30-21ksoftirqd/207:55:012
2694919994116,6cyclictest30-21ksoftirqd/208:31:292
2694919994116,6cyclictest30-21ksoftirqd/208:31:292
269491499417,6cyclictest131rcu_preempt11:15:021
269491499413,4cyclictest131rcu_preempt12:10:091
2694914994128,9cyclictest3467602-21systemd-journal07:38:521
2694914994128,10cyclictest23-21ksoftirqd/110:41:591
2694914994127,11cyclictest1-21systemd08:05:001
269491499412,4cyclictest131rcu_preempt10:32:501
2694912994133,5cyclictest16950irq/224-800000007:40:000
269491299412,3cyclictest131rcu_preempt08:55:010
269492299403,3cyclictest131rcu_preempt10:59:223
2694922994030,7cyclictest37-21ksoftirqd/312:32:273
2694922994030,7cyclictest2781141-21cron12:00:003
2694922994027,11cyclictest37-21ksoftirqd/307:30:013
2694922994026,9cyclictest37-21ksoftirqd/310:50:003
2694922994026,11cyclictest37-21ksoftirqd/310:35:013
269492299402,3cyclictest131rcu_preempt09:43:193
269492299402,3cyclictest131rcu_preempt07:10:023
269492299401,3cyclictest131rcu_preempt09:39:393
2694914994029,8cyclictest23-21ksoftirqd/112:26:181
2694914994027,10cyclictest23-21ksoftirqd/111:31:181
2694914994024,11cyclictest23-21ksoftirqd/110:36:301
2694912994015,18cyclictest16850irq/134-800000007:27:170
269491299401,3cyclictest131rcu_preempt08:30:010
269492299399,5cyclictest131rcu_preempt07:40:023
269492299393,3cyclictest131rcu_preempt09:05:013
2694922993927,9cyclictest37-21ksoftirqd/309:30:143
2694922993926,9cyclictest37-21ksoftirqd/311:00:443
269492299392,5cyclictest131rcu_preempt11:33:043
2694922993924,11cyclictest37-21ksoftirqd/310:20:033
2694922993920,13cyclictest275-21dbus-daemon07:35:013
2694922993919,16cyclictest37-21ksoftirqd/309:10:003
2694919993920,4cyclictest30-21ksoftirqd/207:45:012
269491499398,6cyclictest131rcu_preempt08:30:011
269491499392,5cyclictest131rcu_preempt09:55:021
2694914993924,11cyclictest23-21ksoftirqd/108:10:021
2694914993924,11cyclictest23-21ksoftirqd/108:10:021
2694914993922,12cyclictest309-21in:imuxsock11:42:281
2694914993922,12cyclictest309-21in:imuxsock11:42:281
269491499391,6cyclictest131rcu_preempt08:20:021
269491299393,3cyclictest131rcu_preempt08:25:000
2694912993929,7cyclictest12-21ksoftirqd/007:15:020
2694912993911,9cyclictest2695774-21date07:25:020
2694922993827,8cyclictest2700517-21sshd08:58:293
2694922993827,8cyclictest2700517-21sshd08:58:293
2694922993825,10cyclictest37-21ksoftirqd/308:29:523
2694922993824,10cyclictest275-21dbus-daemon08:10:013
2694922993824,10cyclictest275-21dbus-daemon08:10:013
2694922993810,5cyclictest131rcu_preempt09:25:013
2694919993815,6cyclictest291rcuc/207:30:022
269491499385,6cyclictest131rcu_preempt08:25:001
269491499385,24cyclictest2700281-21modprobe08:55:001
2694914993819,13cyclictest275-21dbus-daemon07:35:011
2694914993813,15cyclictest0-21swapper/107:20:011
2694914993811,5cyclictest131rcu_preempt08:45:291
269491299383,30cyclictest0-21swapper/007:50:470
2694912993828,7cyclictest2698183-21sshd08:12:290
269492299379,5cyclictest131rcu_preempt07:50:033
2694922993729,6cyclictest37-21ksoftirqd/308:00:033
2694922993727,5cyclictest309-21in:imuxsock08:20:013
2694922993726,9cyclictest37-21ksoftirqd/312:23:083
2694922993725,10cyclictest37-21ksoftirqd/312:28:323
2694922993723,10cyclictest37-21ksoftirqd/308:32:293
2694922993723,10cyclictest37-21ksoftirqd/308:32:293
269492299371,3cyclictest131rcu_preempt10:40:033
269492299371,32cyclictest48-21kcompactd008:10:293
2694912993731,3cyclictest16950irq/224-800000007:41:180
269491299371,4cyclictest2697702-21modprobe08:05:000
2694922993625,9cyclictest37-21ksoftirqd/310:19:103
2694922993625,9cyclictest37-21ksoftirqd/310:19:103
269492299362,3cyclictest131rcu_preempt08:05:003
2694922993623,8cyclictest37-21ksoftirqd/308:50:013
269491999364,3cyclictest131rcu_preempt07:15:012
2694919993627,4cyclictest30-21ksoftirqd/207:35:032
2694922993525,7cyclictest2697166-21cron07:55:003
2694922993523,8cyclictest37-21ksoftirqd/311:07:283
269491499352,4cyclictest131rcu_preempt07:10:021
2694914993515,6cyclictest221rcuc/107:55:011
269491299359,17cyclictest2694910-21ls07:10:010
269492299344,6cyclictest0-21swapper/308:25:003
269492299342,3cyclictest131rcu_preempt08:45:013
2694922993325,5cyclictest2695589-21latency_hist07:20:023
269492299332,2cyclictest0-21swapper/307:45:023
269491999332,6cyclictest0-21swapper/207:10:022
2694922993222,7cyclictest37-21ksoftirqd/308:35:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional