You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 16:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot4s.osadl.org (updated Wed Jul 15, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21042995810,9cyclictest131rcu_preempt09:45:121
2105999576,46cyclictest35966irq/58-eth011:25:123
21059995747,7cyclictest35966irq/58-eth012:35:153
2104299522,35cyclictest15853-21ssh10:45:141
2105999474,25cyclictest35966irq/58-eth010:10:123
21059994617,18cyclictest35966irq/58-eth009:15:123
2105999452,29cyclictest35966irq/58-eth007:15:123
2105199451,25cyclictest0-21swapper/209:45:152
2105999423,21cyclictest35966irq/58-eth011:00:123
21059994225,12cyclictest35966irq/58-eth010:00:133
21059994220,11cyclictest35966irq/58-eth009:20:153
2105999412,25cyclictest35966irq/58-eth010:05:143
21051994114,9cyclictest27-21ksoftirqd/211:50:142
21051994114,9cyclictest27-21ksoftirqd/211:50:132
2103499414,30cyclictest6883-21ssh10:40:140
2105999402,21cyclictest35966irq/58-eth011:50:113
2105999402,21cyclictest35966irq/58-eth011:50:113
21059994021,11cyclictest35966irq/58-eth009:25:133
2104299404,22cyclictest21-21ksoftirqd/107:15:131
2104299401,30cyclictest19173-21ssh12:35:141
2105999391,26cyclictest35966irq/58-eth010:25:133
21051993925,9cyclictest27-21ksoftirqd/210:11:412
2105199392,31cyclictest18747-21sh11:04:412
2105999383,20cyclictest35966irq/58-eth011:35:143
2105199382,24cyclictest1445-21ssh10:55:152
2104299382,27cyclictest25060-21ssh09:20:151
2103499381,33cyclictest6467-21ssh12:10:110
2103499381,19cyclictest0-21swapper/010:15:130
2105199372,25cyclictest17506-21ssh11:40:132
2105199362,18cyclictest14757-21ssh12:14:412
2104299363,27cyclictest22917-21ssh11:25:121
2104299357,11cyclictest21-21ksoftirqd/109:50:141
2103499353,25cyclictest7081-21ssh09:10:120
21034993523,6cyclictest27298-21apt-get07:50:130
21034993523,6cyclictest27298-21apt-get07:50:130
2104299346,5cyclictest131rcu_preempt11:05:171
2104299341,28cyclictest20975-21ssh12:00:121
2103499342,8cyclictest1566-21ssh10:55:170
21059993318,9cyclictest0-21swapper/312:15:133
2105199335,15cyclictest27346irq/40-dwc2_hso07:35:142
2105199334,7cyclictest131rcu_preempt08:15:142
2105199333,22cyclictest27-21ksoftirqd/210:00:132
2104299336,23cyclictest26748-21ssh10:15:141
21059993213,13cyclictest35966irq/58-eth012:00:143
2105999321,16cyclictest35966irq/58-eth010:30:113
2104299326,18cyclictest21-21ksoftirqd/112:30:111
21034993223,3cyclictest17671-21apt-get07:40:130
21034993217,10cyclictest2104-21apt-get09:05:160
2105199312,4cyclictest131rcu_preempt09:10:122
2105199312,24cyclictest24448-21ssh09:35:402
2105199312,24cyclictest131rcu_preempt10:08:572
2104299312,22cyclictest23563-21ssh10:11:401
2103499315,5cyclictest5245-21ntpq09:25:270
2105999303,13cyclictest35966irq/58-eth011:20:123
2105199302,22cyclictest29584-21ssh10:34:402
2105999294,14cyclictest0-21swapper/310:15:153
2105199295,14cyclictest27346irq/40-dwc2_hso08:10:122
2105199292,21cyclictest0-21swapper/207:15:152
2105199291,23cyclictest15456-21ssh12:15:152
2104299297,17cyclictest21-21ksoftirqd/110:25:141
2103499294,5cyclictest12470-21find11:35:300
2105199282,22cyclictest28487-21ssh11:10:152
2103499285,6cyclictest10829-1kworker/u9:210:00:200
2103499284,5cyclictest12-21ksoftirqd/010:35:150
2103499282,5cyclictest15884-21memory10:25:250
2103499282,4cyclictest12-21ksoftirqd/012:05:270
2103499282,20cyclictest1017-21ssh11:10:420
2103499281,7cyclictest21878-21ssh09:16:470
21034992817,7cyclictest6683-21diskmemload09:35:150
2103499281,21cyclictest12019-21ssh11:55:140
21034992812,12cyclictest3031-21ssh11:50:140
21034992812,12cyclictest3031-21ssh11:50:130
2103499281,14cyclictest8723-21ssh10:05:140
2105199273,19cyclictest0-21swapper/209:57:402
2105199272,21cyclictest16301-21ssh09:31:402
21051992719,4cyclictest27346irq/40-dwc2_hso09:50:132
2105199271,15cyclictest24881-21ssh10:50:152
2104299271,22cyclictest12003-21ssh11:55:141
2103499274,4cyclictest12-21ksoftirqd/011:40:190
2103499273,4cyclictest21440-21ntp_states10:10:270
2103499272,20cyclictest481kcompactd011:30:160
21034992720,3cyclictest18858-21apt-get08:15:140
2105999263,15cyclictest0-21swapper/309:35:153
2105199265,14cyclictest27346irq/40-dwc2_hso09:20:152
2105199262,17cyclictest27-21ksoftirqd/211:35:132
2105199261,20cyclictest27346irq/40-dwc2_hso11:20:132
2105199261,13cyclictest22921-21sh11:25:122
2104299263,20cyclictest16038-21sh09:15:121
2104299262,18cyclictest25492-21ssh11:44:191
21042992617,7cyclictest21-21ksoftirqd/110:05:161
2103499269,10cyclictest22896-21apt-get11:25:110
2103499268,13cyclictest0-21swapper/009:50:140
2103499265,8cyclictest12-21ksoftirqd/009:40:190
2103499265,4cyclictest12-21ksoftirqd/007:20:120
2103499263,6cyclictest12-21ksoftirqd/012:20:220
2103499263,6cyclictest12-21ksoftirqd/012:20:210
2103499262,19cyclictest19493-21ssh11:05:160
2103499262,18cyclictest13305-21sh09:49:400
21034992619,3cyclictest26004-21apt-get07:15:130
2103499261,7cyclictest14240-21ps09:30:260
2103499261,7cyclictest12-21ksoftirqd/012:36:020
21034992617,5cyclictest10391-21apt-get11:00:110
2103499261,6cyclictest13933-21/usr/sbin/munin11:20:210
21034992613,9cyclictest10157-21apt-get12:30:110
2105999253,13cyclictest0-21swapper/310:39:393
2105199252,9cyclictest0-21swapper/208:20:142
2105199252,17cyclictest18152-21ssh10:26:392
2105199252,16cyclictest27346irq/40-dwc2_hso07:10:132
2105199251,19cyclictest5667-21ssh10:39:032
2105199251,18cyclictest26001-21ssh12:00:422
2104299258,5cyclictest0-21swapper/109:14:461
2104299256,10cyclictest21-21ksoftirqd/110:35:141
2104299251,8cyclictest131rcu_preempt07:50:141
2104299251,8cyclictest131rcu_preempt07:50:141
2104299251,7cyclictest0-21swapper/108:40:131
2104299251,7cyclictest0-21swapper/108:40:121
21042992515,7cyclictest21-21ksoftirqd/111:00:111
2104299250,23cyclictest4978-21ssh11:15:131
2103499254,6cyclictest25528-21cut08:55:190
2103499254,4cyclictest12-21ksoftirqd/010:20:160
2103499253,5cyclictest15836-21apt-get10:45:120
2103499253,5cyclictest12-21ksoftirqd/008:25:300
2103499253,5cyclictest12-21ksoftirqd/007:45:240
2103499251,6cyclictest19612-21unixbench_singl12:15:300
21034992512,8cyclictest26484-21ssh11:45:140
2103499250,9cyclictest29880-21ssh10:50:460
2105999247,7cyclictest35966irq/58-eth012:30:113
2105999243,13cyclictest0-21swapper/309:49:393
21059992415,5cyclictest0-21swapper/311:10:143
21059992413,7cyclictest35966irq/58-eth009:55:133
2105199243,11cyclictest27346irq/40-dwc2_hso09:00:132
2105199242,8cyclictest0-21swapper/207:40:132
2105199242,14cyclictest27346irq/40-dwc2_hso08:30:102
2105199241,3cyclictest0-21swapper/209:25:182
2105199240,21cyclictest6907-21ssh10:40:142
2104299249,5cyclictest21-21ksoftirqd/112:25:201
2104299244,6cyclictest131rcu_preempt09:35:131
2104299243,16cyclictest21-21ksoftirqd/111:45:141
2104299242,18cyclictest31994-21ssh11:30:161
2104299241,17cyclictest21-21ksoftirqd/110:30:101
2103499244,4cyclictest12-21ksoftirqd/011:15:270
2103499242,6cyclictest12-21ksoftirqd/009:20:210
2103499242,4cyclictest20701-21irqstats08:50:210
2103499242,4cyclictest12-21ksoftirqd/012:00:200
21034992417,2cyclictest3223-21apt-get07:25:130
2103499241,6cyclictest12-21ksoftirqd/012:25:240
2103499240,7cyclictest448-21grep09:00:200
2105999236,11cyclictest0-21swapper/310:58:393
2105999233,12cyclictest0-21swapper/311:55:153
2105999233,10cyclictest0-21swapper/310:24:013
2105999232,10cyclictest0-21swapper/311:44:413
21059992313,7cyclictest0-21swapper/309:50:153
2105199232,6cyclictest0-21swapper/208:00:122
2105199232,16cyclictest0-21swapper/210:18:592
2105199232,15cyclictest27346irq/40-dwc2_hso12:35:152
21051992318,3cyclictest261rcuc/211:15:132
21051992311,5cyclictest20844-21ssh10:45:422
2105199230,8cyclictest2329-21kthreadcore11:30:252
2105199230,20cyclictest27620-21sed07:50:192
2105199230,20cyclictest27620-21sed07:50:182
2104299232,15cyclictest15975-21ssh11:38:401
2104299231,18cyclictest2259-21cut09:05:181
21042992311,3cyclictest6683-21diskmemload10:53:161
2103499234,6cyclictest12-21ksoftirqd/008:40:210
2103499234,6cyclictest12-21ksoftirqd/008:40:210
2103499234,4cyclictest12-21ksoftirqd/008:35:210
2103499233,5cyclictest12-21ksoftirqd/009:58:470
2103499233,5cyclictest12-21ksoftirqd/007:10:010
2103499233,4cyclictest32319-21sed07:55:190
2103499233,4cyclictest12-21ksoftirqd/010:30:110
2103499233,4cyclictest12-21ksoftirqd/007:30:180
2103499231,17cyclictest14245-21uniq08:10:160
2103499230,8cyclictest27821-21cat08:20:300
2105999223,13cyclictest0-21swapper/311:49:423
2105999223,10cyclictest35966irq/58-eth012:25:133
2105999222,17cyclictest35966irq/58-eth011:15:133
2105199225,14cyclictest0-21swapper/207:45:122
2105199223,8cyclictest27-21ksoftirqd/212:05:112
2105199222,6cyclictest0-21swapper/211:05:402
2105199222,15cyclictest0-21swapper/210:21:402
2105199221,9cyclictest0-21swapper/211:45:152
21051992212,8cyclictest0-21swapper/209:15:122
2105199221,12cyclictest28486-21ssh09:40:142
21051992210,5cyclictest16334-21ssh12:32:072
2104299228,10cyclictest31619-21basename10:00:021
2104299224,10cyclictest0-21swapper/112:20:121
2104299224,10cyclictest0-21swapper/112:20:111
2104299221,2cyclictest131rcu_preempt11:20:281
2104299221,18cyclictest30145-21uniq12:05:161
2104299221,18cyclictest1447-21rm10:55:161
2104299221,15cyclictest3385-21ssh10:20:141
2103499223,5cyclictest12-21ksoftirqd/008:00:170
2103499223,4cyclictest12-21ksoftirqd/007:10:110
21034992216,3cyclictest855-21apt-get08:30:100
21034992212,6cyclictest12874-21apt-get07:35:130
2103499220,4cyclictest13862-21munin-run08:10:020
2105999214,10cyclictest35966irq/58-eth012:20:133
2105999214,10cyclictest35966irq/58-eth012:20:123
2105999213,12cyclictest35966irq/58-eth010:45:123
2105999213,11cyclictest0-21swapper/309:40:153
2105199213,9cyclictest13979-21cut11:55:222
2105199212,6cyclictest0-21swapper/208:55:132
21051992114,5cyclictest2793-21ssh12:25:212
2105199211,15cyclictest20295-21ls08:50:202
2105199211,12cyclictest6683-21diskmemload12:20:152
2105199211,12cyclictest6683-21diskmemload12:20:142
2104299218,3cyclictest131rcu_preempt09:25:181
21042992113,4cyclictest21-21ksoftirqd/107:10:241
21042992110,8cyclictest28266-21cron08:25:011
21042992110,7cyclictest31917-21sed11:10:221
2105999202,13cyclictest0-21swapper/311:09:103
21059992010,7cyclictest35966irq/58-eth009:10:123
2105199206,11cyclictest0-21swapper/208:35:172
21051992013,5cyclictest0-21swapper/208:40:282
21051992013,5cyclictest0-21swapper/208:40:272
2105199201,15cyclictest3208-21acpi07:25:122
2105199201,14cyclictest721-21cat08:30:022
2105199201,12cyclictest12747-21ls07:35:012
2105199201,11cyclictest5108-21kthreadcore09:05:242
2105199201,10cyclictest18405-21kthreadcore08:45:232
2105199201,10cyclictest12057-21kthreadcore08:05:242
2104299202,14cyclictest844-21munin-node08:15:141
2104299201,4cyclictest131rcu_preempt09:30:111
2104299201,10cyclictest131rcu_preempt09:42:051
2104299200,9cyclictest131rcu_preempt12:14:261
2103499201,6cyclictest15437-21fschecks_time08:45:160
2105999199,8cyclictest0-21swapper/312:10:523
2105999199,8cyclictest0-21swapper/309:32:033
2105999193,10cyclictest35966irq/58-eth007:30:143
2105999192,12cyclictest0-21swapper/310:44:413
21059991913,4cyclictest0-21swapper/312:05:203
2105999191,13cyclictest0-21swapper/308:00:163
2105199191,9cyclictest30994-21fschecks_time07:20:162
2105199191,8cyclictest3004-21missed_timers07:55:242
2104299199,3cyclictest0-21swapper/108:00:261
2104299193,7cyclictest131rcu_preempt07:30:221
2104299193,3cyclictest131rcu_preempt12:15:171
2104299192,9cyclictest131rcu_preempt10:00:271
2104299191,7cyclictest5133-21kthreadcore11:50:211
2104299191,7cyclictest5133-21kthreadcore11:50:201
21042991912,5cyclictest23918-21unixbench_multi08:50:311
21042991911,6cyclictest17415-21cron07:40:011
21042991911,5cyclictest7642-21kthreadcore10:40:211
2104299190,12cyclictest1020-21cut08:30:141
2105999188,8cyclictest0-21swapper/311:30:053
2105999188,6cyclictest35966irq/58-eth010:50:153
2105999182,11cyclictest0-21swapper/307:35:143
2105999182,10cyclictest0-21swapper/308:20:143
2105999182,10cyclictest0-21swapper/307:25:133
2105999182,10cyclictest0-21swapper/307:20:143
2105999181,10cyclictest0-21swapper/307:45:023
2105199181,13cyclictest21052-21latency_hist07:10:022
2104299189,6cyclictest7069-21grep07:25:281
2104299182,2cyclictest131rcu_preempt08:25:201
2104299181,6cyclictest22992-21tr07:45:171
21042991812,3cyclictest14161-21cpuspeed08:10:151
2104299181,13cyclictest0-21swapper/107:20:151
21042991810,5cyclictest21-21ksoftirqd/108:05:131
21042991810,5cyclictest0-21swapper/107:55:131
2105999171,9cyclictest0-21swapper/307:55:023
2105999171,9cyclictest0-21swapper/307:55:013
2104299172,2cyclictest131rcu_preempt08:55:151
2104299172,10cyclictest0-21swapper/107:40:131
2104299171,12cyclictest15377-21diskstats08:45:171
2105999165,5cyclictest0-21swapper/308:45:143
2105999161,12cyclictest0-21swapper/307:55:253
2105999161,10cyclictest0-21swapper/308:05:153
2104299167,3cyclictest0-21swapper/109:00:121
2104299162,4cyclictest131rcu_preempt08:35:301
2105999152,9cyclictest0-21swapper/308:30:103
2105999151,8cyclictest0-21swapper/308:25:283
2105999142,9cyclictest0-21swapper/308:40:123
2105999142,9cyclictest0-21swapper/308:40:123
2105999141,8cyclictest0-21swapper/309:05:273
2105999141,7cyclictest0-21swapper/308:15:223
2105999141,11cyclictest0-21swapper/307:50:013
2104299141,10cyclictest21061-21latency_hist07:10:011
2105999138,3cyclictest0-21swapper/308:51:153
2105999131,7cyclictest0-21swapper/308:55:153
2105999131,10cyclictest0-21swapper/308:15:013
2105999131,10cyclictest0-21swapper/307:10:223
2105999122,6cyclictest0-21swapper/309:00:123
2105999121,5cyclictest0-21swapper/308:35:143
2105999111,7cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional