You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 17:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed May 27, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1937999653,49cyclictest7813-21ssh10:05:131
19379996512,6cyclictest21-21ksoftirqd/112:05:141
1937999612,40cyclictest6374-21ssh12:10:121
1936499523,40cyclictest11613-21ssh11:55:110
19388995110,7cyclictest131rcu_preempt07:20:102
19379994817,23cyclictest910-21ssh12:25:131
19379994817,23cyclictest910-21ssh12:25:131
1936499481,39cyclictest14916-21ssh09:15:120
1939999473,29cyclictest35966irq/58-eth012:25:133
1939999473,29cyclictest35966irq/58-eth012:25:123
1937999473,7cyclictest131rcu_preempt07:40:121
1937999471,39cyclictest20278-21ssh10:30:101
1937999432,34cyclictest16756-21ssh10:10:101
1936499433,29cyclictest18503-21sh09:35:110
1936499432,29cyclictest13153-21ssh09:50:110
1936499423,31cyclictest22121-21ssh09:55:130
1936499421,34cyclictest25961-21sh11:45:130
1936499421,33cyclictest27498-21ssh09:40:120
1936499411,33cyclictest6007-21sh10:40:120
19388993612,9cyclictest27-21ksoftirqd/209:10:122
19364993618,11cyclictest151rcuc/012:20:140
1936499361,4cyclictest131rcu_preempt12:15:110
19399993517,12cyclictest35966irq/58-eth010:35:123
1936499355,17cyclictest31648-21kworker/u8:011:35:130
19399993427,4cyclictest35966irq/58-eth011:40:123
1939999333,20cyclictest35966irq/58-eth009:05:123
1939999333,20cyclictest35966irq/58-eth009:05:123
19388993313,12cyclictest27-21ksoftirqd/212:30:112
1936499332,23cyclictest25325-21ssh11:44:430
1939999324,18cyclictest35966irq/58-eth010:45:113
19399993220,9cyclictest0-21swapper/312:30:123
1936499323,22cyclictest31395-21ssh11:30:120
19399993121,8cyclictest0-21swapper/312:35:133
1938899313,17cyclictest27346irq/40-dwc2_hso07:50:122
1937999302,22cyclictest29820-21ssh10:51:331
1937999301,20cyclictest15030-21ssh10:45:111
1936499302,19cyclictest27777-21ssh11:10:120
1936499302,19cyclictest18911-21ssh12:35:130
1937999291,4cyclictest6331-21apt-get07:30:121
1936499292,20cyclictest9781-21ssh11:00:120
1936499291,22cyclictest9921-21ssh12:30:130
1938899282,19cyclictest0-21swapper/208:10:132
19388992820,3cyclictest27-21ksoftirqd/210:05:122
19388992813,6cyclictest27368-21cron11:10:012
1937999283,4cyclictest131rcu_preempt07:50:121
1937999282,21cyclictest7285-21ssh10:57:331
1936499282,21cyclictest5232-21rm10:39:350
1936499282,20cyclictest2603-21ssh11:50:130
1936499282,18cyclictest31194-21ssh10:00:140
1938899273,9cyclictest0-21swapper/210:35:322
1938899273,19cyclictest27346irq/40-dwc2_hso10:45:112
1938899272,16cyclictest27346irq/40-dwc2_hso12:00:132
19388992714,10cyclictest14656-21cron10:45:012
19388992710,10cyclictest27-21ksoftirqd/210:15:122
1936499271,20cyclictest20779-21ssh12:00:130
1939999263,16cyclictest0-21swapper/310:57:333
1938899263,7cyclictest0-21swapper/210:57:322
1938899262,10cyclictest0-21swapper/207:40:132
19388992618,5cyclictest31864-21ssh11:10:282
1936499263,17cyclictest11336-21ssh10:25:130
1936499262,17cyclictest696-21ssh10:55:140
1939999253,11cyclictest0-21swapper/311:02:333
1939999252,17cyclictest0-21swapper/311:13:333
19388992518,3cyclictest24096-21ssh10:50:112
1938899251,6cyclictest2517366ssh09:37:422
1938899251,6cyclictest2494066ssh12:36:572
1938899251,19cyclictest29857-21ssh12:05:132
1937999252,16cyclictest12161-21kworker/1:008:10:121
19379992515,7cyclictest27773-21ssh10:15:191
1937999251,3cyclictest131rcu_preempt11:25:141
1937999251,16cyclictest23906-21ssh09:20:121
19364992519,4cyclictest12-21ksoftirqd/009:20:280
1939999243,10cyclictest0-21swapper/311:33:333
1938899242,17cyclictest18701-21ssh10:28:332
1938899242,16cyclictest27346irq/40-dwc2_hso08:30:122
1938899241,6cyclictest190966rm11:49:462
1937999242,6cyclictest0-21swapper/111:34:331
1937999242,17cyclictest16326-21ssh11:02:341
19379992412,4cyclictest21-21ksoftirqd/109:55:141
19364992413,8cyclictest0-21swapper/008:25:170
19364992413,5cyclictest12-21ksoftirqd/009:25:330
1936499241,19cyclictest4460-21ssh11:15:130
1939999237,11cyclictest0-21swapper/311:05:103
1938899234,15cyclictest27346irq/40-dwc2_hso07:10:122
1938899234,15cyclictest27346irq/40-dwc2_hso07:10:112
1938899233,15cyclictest0-21swapper/211:33:332
1938899233,14cyclictest27346irq/40-dwc2_hso08:45:122
1938899233,13cyclictest27346irq/40-dwc2_hso12:25:122
1938899233,13cyclictest27346irq/40-dwc2_hso12:25:122
1938899233,11cyclictest21023-21usage_Anybus10:10:292
1938899232,6cyclictest0-21swapper/211:00:332
1938899231,9cyclictest1305966ssh11:20:012
1938899231,7cyclictest543866diskmemload11:37:462
1938899231,7cyclictest2228766ssh11:41:042
19388992312,8cyclictest28477-21tr09:40:182
1938899230,6cyclictest2035666rm09:53:232
1937999232,4cyclictest131rcu_preempt09:50:251
19379992314,6cyclictest4322-21ssh11:15:031
1936499238,10cyclictest16909-21fschecks_count10:10:150
1936499231,17cyclictest16123-21cut07:40:140
1939999228,10cyclictest0-21swapper/310:25:213
1939999225,12cyclictest0-21swapper/310:52:323
19399992213,6cyclictest35966irq/58-eth009:45:133
19399992210,9cyclictest0-21swapper/309:16:463
1938899226,9cyclictest131rcu_preempt09:20:132
1938899226,6cyclictest0-21swapper/209:00:282
1938899221,9cyclictest887066ssh09:29:432
1938899221,7cyclictest591566ssh10:20:242
1938899221,6cyclictest2949166ssh12:20:502
1938899221,6cyclictest2611566ssh10:31:402
1938899221,6cyclictest2044466ssh09:16:192
1938899221,6cyclictest1683566ssh09:33:292
1938899221,6cyclictest1683566ssh09:33:292
19388992212,8cyclictest0-21swapper/208:00:312
19388992212,8cyclictest0-21swapper/208:00:302
1938899221,12cyclictest20330-21ssh12:00:012
1938899221,11cyclictest25854-21sh09:55:262
1937999228,4cyclictest131rcu_preempt11:35:181
1937999227,9cyclictest21-21ksoftirqd/108:25:141
1937999221,8cyclictest27456-21munin-node09:40:101
19379992213,6cyclictest28640-21ssh11:45:201
1937999221,16cyclictest7867-21ssh09:45:261
1937999221,10cyclictest0-21swapper/112:24:061
1936499229,4cyclictest12-21ksoftirqd/010:09:050
1936499228,6cyclictest12-21ksoftirqd/008:05:170
1936499228,5cyclictest12-21ksoftirqd/010:54:110
1936499227,9cyclictest12-21ksoftirqd/008:15:160
1936499223,3cyclictest131rcu_preempt10:18:110
19364992214,6cyclictest15968-21kthreadcore10:45:180
1936499221,19cyclictest13509-21apt-key08:45:010
1939999213,10cyclictest0-21swapper/310:30:113
1939999212,15cyclictest35966irq/58-eth008:50:143
19399992112,7cyclictest0-21swapper/309:36:123
1938899219,9cyclictest17653-21ssh12:15:192
1938899219,5cyclictest10889-21sshd07:33:322
1938899218,5cyclictest22859-21cut11:25:202
1938899216,7cyclictest27-21ksoftirqd/211:20:222
1938899215,13cyclictest31400-21fschecks_time10:00:162
1938899213,9cyclictest9478-21ps08:05:212
1938899211,6cyclictest623366ssh12:10:042
1938899211,6cyclictest28077-21cron09:00:022
1938899211,6cyclictest1162166ssh09:48:382
1938899211,5cyclictest30378-21/usr/sbin/munin07:55:322
1938899211,17cyclictest14851-21timerandwakeup07:35:282
1938899211,17cyclictest10544-21kthreadcore08:40:202
1938899211,10cyclictest22338-21timerwakeupswit08:50:282
1937999213,3cyclictest131rcu_preempt09:35:211
1937999211,6cyclictest20621-21ntp_states11:40:241
19379992113,4cyclictest21-21ksoftirqd/108:15:121
19379992112,6cyclictest5959-21ssh10:20:241
19379992112,6cyclictest23861-21ssh12:35:401
1937999211,16cyclictest20769-21ssh12:00:131
1937999211,16cyclictest17149-21ssh11:20:271
1937999211,15cyclictest13875-21ls08:45:131
1937999211,10cyclictest0-21swapper/112:32:051
1937999211,10cyclictest0-21swapper/110:38:351
1936499218,10cyclictest25924-21grep08:20:280
1936499215,9cyclictest12-21ksoftirqd/011:28:590
1936499215,7cyclictest12-21ksoftirqd/010:33:060
1936499213,13cyclictest4152-21ssh09:45:120
19364992112,4cyclictest12-21ksoftirqd/009:14:470
1936499211,17cyclictest2554-21cut10:20:170
1936499211,16cyclictest9515-21ssh09:30:130
1936499211,16cyclictest9515-21ssh09:30:120
1939999209,5cyclictest0-21swapper/311:15:133
1939999208,9cyclictest0-21swapper/309:54:043
1939999204,10cyclictest35966irq/58-eth012:15:113
19399992012,6cyclictest0-21swapper/310:20:273
19399992011,7cyclictest0-21swapper/312:05:173
1939999201,12cyclictest0-21swapper/310:10:113
1938899208,9cyclictest22617-21cut08:20:162
1938899204,14cyclictest17952-21kthreadcore08:15:192
1938899203,14cyclictest28991-21grep07:20:012
1938899202,9cyclictest23689-21kthreadcore07:45:212
1938899202,8cyclictest31638-21latency_hist08:30:022
1938899202,10cyclictest4331-21hwlatdetect08:35:172
1938899202,10cyclictest2817-21cut11:50:142
1938899201,9cyclictest1643-21fschecks_count07:25:172
1938899201,10cyclictest3170-21kthreadcore09:05:202
1938899201,10cyclictest3170-21kthreadcore09:05:202
1937999205,4cyclictest25257-21kthreadcore07:15:201
1937999201,9cyclictest0-21swapper/110:43:181
1937999201,8cyclictest11448-21diskstats10:25:151
19379992012,6cyclictest2885-21ssh11:13:431
19379992012,5cyclictest5789-21ssh09:10:051
19379992011,6cyclictest4071-21sed09:25:241
1936499208,4cyclictest12-21ksoftirqd/012:12:070
1936499207,6cyclictest12-21ksoftirqd/008:45:260
1936499205,4cyclictest12-21ksoftirqd/012:09:070
1936499204,4cyclictest12-21ksoftirqd/007:25:230
1936499204,13cyclictest0-21swapper/011:20:120
1936499202,4cyclictest12-21ksoftirqd/009:05:270
1936499202,4cyclictest12-21ksoftirqd/009:05:260
1936499202,12cyclictest12-21ksoftirqd/007:30:140
19364992014,4cyclictest0-21swapper/008:10:150
19364992013,4cyclictest0-21swapper/009:00:130
19364992011,6cyclictest0-21swapper/007:45:270
1939999199,8cyclictest0-21swapper/311:27:443
1939999199,6cyclictest0-21swapper/311:59:283
1939999198,8cyclictest0-21swapper/311:49:373
1939999198,8cyclictest0-21swapper/309:10:573
1939999198,6cyclictest0-21swapper/310:44:253
1939999192,11cyclictest0-21swapper/308:00:143
1939999192,11cyclictest0-21swapper/308:00:143
19399991913,4cyclictest0-21swapper/310:15:193
19399991910,6cyclictest0-21swapper/311:50:153
19399991910,5cyclictest35966irq/58-eth009:55:133
1938899191,9cyclictest19352-21cyclictest07:10:012
1937999197,9cyclictest18745-21cat11:05:091
1937999194,7cyclictest131rcu_preempt12:18:001
1937999193,10cyclictest0-21swapper/108:30:131
1937999192,8cyclictest0-21swapper/111:59:511
1937999192,8cyclictest0-21swapper/110:00:561
19379991912,5cyclictest10249-21ssh11:53:511
19379991911,6cyclictest12165-21sshd08:07:321
19379991910,6cyclictest13969-21ssh09:30:301
19379991910,6cyclictest13969-21ssh09:30:301
1937999190,5cyclictest2633-21basename08:00:011
1936499198,4cyclictest12-21ksoftirqd/012:30:010
1936499198,4cyclictest12-21ksoftirqd/012:30:010
1936499197,4cyclictest12-21ksoftirqd/011:05:200
1936499197,4cyclictest12-21ksoftirqd/007:20:010
1936499196,2cyclictest131rcu_preempt08:35:220
19364991914,3cyclictest0-21swapper/008:50:140
19364991913,4cyclictest0-21swapper/007:50:250
19364991913,4cyclictest0-21swapper/007:10:240
19364991913,4cyclictest0-21swapper/007:10:240
1939999189,6cyclictest35966irq/58-eth011:35:133
1939999182,10cyclictest0-21swapper/308:45:033
1939999182,10cyclictest0-21swapper/308:30:143
19399991813,3cyclictest33-21ksoftirqd/309:25:303
19399991812,4cyclictest0-21swapper/309:22:293
1939999181,13cyclictest35966irq/58-eth008:35:123
1939999181,13cyclictest0-21swapper/309:45:023
19399991810,6cyclictest0-21swapper/312:12:593
19399991810,6cyclictest0-21swapper/311:24:153
19399991810,6cyclictest0-21swapper/310:00:443
19399991810,6cyclictest0-21swapper/309:34:273
19399991810,6cyclictest0-21swapper/309:34:273
1937999188,8cyclictest28417-21apt-get09:00:131
1937999188,7cyclictest20784-21apt-get07:45:111
1937999183,11cyclictest0-21swapper/109:05:131
1937999183,11cyclictest0-21swapper/109:05:121
1937999181,9cyclictest22279-21cut08:20:151
1937999181,9cyclictest14601-21ntp_offset07:35:241
19379991810,6cyclictest21577-21ssh09:17:421
1936499181,7cyclictest7384-21munin-run08:05:010
1936499181,7cyclictest7384-21munin-run08:05:010
19364991812,4cyclictest0-21swapper/007:20:240
1936499181,13cyclictest31722-21apt-get08:30:120
1939999179,5cyclictest35966irq/58-eth010:05:113
1939999177,8cyclictest0-21swapper/312:21:083
1939999172,10cyclictest0-21swapper/307:40:133
19399991712,3cyclictest33-21ksoftirqd/307:20:253
19399991711,4cyclictest0-21swapper/312:00:263
19399991710,5cyclictest33-21ksoftirqd/307:35:273
1937999179,5cyclictest20459-21kthreadcore08:50:201
1937999178,6cyclictest30984-21kthreadcore07:20:191
1937999173,11cyclictest0-21swapper/107:10:121
1937999173,11cyclictest0-21swapper/107:10:121
1936499173,8cyclictest12-21ksoftirqd/008:55:180
1936499172,10cyclictest0-21swapper/007:35:140
19364991711,4cyclictest0-21swapper/007:55:280
1939999164,9cyclictest33-21ksoftirqd/307:30:243
1939999164,8cyclictest35966irq/58-eth008:25:133
19399991611,3cyclictest36066irq/59-eth008:19:193
1937999162,11cyclictest5126-21tr08:00:211
1937999162,11cyclictest5126-21tr08:00:211
1937999161,12cyclictest5145-21kthreadcore08:35:201
19379991610,4cyclictest0-21swapper/108:55:291
1937999160,13cyclictest6107-21munin-run07:30:001
1939999152,10cyclictest0-21swapper/309:00:143
1939999152,10cyclictest0-21swapper/308:55:143
1939999152,10cyclictest0-21swapper/308:05:143
1939999152,10cyclictest0-21swapper/307:55:153
1937999150,2cyclictest131rcu_preempt08:40:121
1936499158,5cyclictest12-21ksoftirqd/007:10:020
1939999141,7cyclictest0-21swapper/307:15:293
1939999141,10cyclictest0-21swapper/307:10:153
1939999141,10cyclictest0-21swapper/307:10:143
1937999141,10cyclictest19398-21latency_hist07:10:021
1939999132,9cyclictest0-21swapper/308:20:133
1939999131,9cyclictest0-21swapper/307:45:263
1939999131,10cyclictest0-21swapper/308:40:163
1939999123,5cyclictest0-21swapper/308:10:123
1939999117,2cyclictest33-21ksoftirqd/307:50:223
1939999111,5cyclictest0-21swapper/307:25:133
193999991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional