You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 08:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sun Jun 14, 2026 00:45:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2390799573,37cyclictest8767-21ssh00:10:172
23921995145,4cyclictest35966irq/58-eth000:10:163
23907994816,26cyclictest27-21ksoftirqd/200:30:112
23907994816,26cyclictest27-21ksoftirqd/200:30:112
2389799423,30cyclictest13503-21ssh21:30:131
2389799412,6cyclictest131rcu_preempt22:45:121
2390799401,20cyclictest23946-21ssh22:30:152
2390799373,13cyclictest30902-21ssh23:10:102
2388699372,27cyclictest32249-21ssh00:05:130
2388699371,27cyclictest17649-21ssh00:15:130
2392199364,22cyclictest35966irq/58-eth022:55:113
23907993620,3cyclictest9398-21diskmemload22:50:142
23921993523,9cyclictest35966irq/58-eth023:35:153
2390799353,19cyclictest9823-21ssh21:10:142
23907993511,12cyclictest131rcu_preempt23:00:122
2392199341,16cyclictest35966irq/58-eth000:15:143
23907993421,6cyclictest27-21ksoftirqd/221:25:122
2389799346,22cyclictest8758-21ssh00:10:161
2389799341,3cyclictest131rcu_preempt22:00:131
2388699341,24cyclictest18837-21ssh21:15:140
2388699341,24cyclictest14974-21ssh22:25:120
2392199333,16cyclictest35966irq/58-eth023:50:153
2390799332,17cyclictest3199-21ssh00:25:132
23907993314,14cyclictest29826-21kworker/2:221:40:132
2388699325,23cyclictest26680-21ssh00:20:120
2392199313,17cyclictest35966irq/58-eth023:30:133
2390799312,18cyclictest27346irq/40-dwc2_hso23:55:132
2388699312,21cyclictest12-21ksoftirqd/022:00:130
2390799305,9cyclictest0-21swapper/221:20:122
2390799304,18cyclictest27-21ksoftirqd/221:00:132
2390799303,19cyclictest27346irq/40-dwc2_hso23:25:132
2390799303,19cyclictest27346irq/40-dwc2_hso23:25:132
23907993017,11cyclictest27-21ksoftirqd/223:45:122
2389799301,21cyclictest9471-21ssh22:40:141
23886993016,8cyclictest0-21swapper/000:00:130
2388699301,23cyclictest7482-21ssh23:15:120
23921992915,8cyclictest35966irq/58-eth021:30:133
2390799297,10cyclictest21173-21sh00:35:142
2390799297,10cyclictest21173-21sh00:35:142
2390799292,20cyclictest395-21ssh22:35:132
2392199284,17cyclictest0-21swapper/323:00:123
2392199282,16cyclictest35966irq/58-eth000:05:133
2390799283,18cyclictest27346irq/40-dwc2_hso19:15:112
2390799283,15cyclictest0-21swapper/221:15:132
2390799282,20cyclictest0-21swapper/219:40:132
2390799282,18cyclictest27346irq/40-dwc2_hso21:50:142
2389799284,14cyclictest29468-21ssh22:15:131
2388699287,13cyclictest12-21ksoftirqd/020:00:110
2388699281,17cyclictest25321-21ssh23:25:120
2388699281,17cyclictest25321-21ssh23:25:120
2392199272,16cyclictest0-21swapper/323:05:133
2390799275,13cyclictest27346irq/40-dwc2_hso20:10:122
23907992714,10cyclictest11797-21cron20:05:012
23897992720,3cyclictest21-21ksoftirqd/121:20:121
23921992616,7cyclictest35966irq/58-eth021:35:133
2392199261,21cyclictest35966irq/58-eth023:45:143
2392199261,14cyclictest0-21swapper/322:45:133
2390799262,16cyclictest27346irq/40-dwc2_hso23:30:132
2390799262,14cyclictest261rcuc/220:20:132
23907992613,9cyclictest3964-21/usr/sbin/munin22:55:282
2390799261,19cyclictest15594-21find19:35:122
2390799261,10cyclictest6746-21cat22:00:332
2392199252,14cyclictest35966irq/58-eth023:15:123
2390799253,17cyclictest0-21swapper/221:35:052
2390799252,18cyclictest10889-21ssh23:35:152
2390799251,6cyclictest22015-21ssh23:05:142
2390799251,12cyclictest19933-21ls23:40:132
2389799252,18cyclictest28792-21apt-get23:45:131
2388699252,18cyclictest14355-21ssh23:55:140
23907992412,9cyclictest14087-21ssh23:17:442
23907992412,6cyclictest261rcuc/220:50:132
2390799241,17cyclictest20491-21cpuspeed22:10:122
2390799241,10cyclictest26033-21sh21:55:162
2390799240,18cyclictest21586-21find20:15:112
2388699241,18cyclictest8044-21ssh21:45:130
2388699241,18cyclictest8044-21ssh21:45:130
2392199232,17cyclictest35966irq/58-eth023:20:153
2392199232,17cyclictest35966irq/58-eth023:20:143
2392199232,17cyclictest35966irq/58-eth021:40:123
2390799232,18cyclictest0-21swapper/220:45:132
2390799232,14cyclictest0-21swapper/220:35:132
23907992312,8cyclictest0-21swapper/219:20:162
2390799231,12cyclictest12621-21tr22:05:192
2388699235,12cyclictest12-21ksoftirqd/023:20:120
2388699235,12cyclictest12-21ksoftirqd/023:20:120
2388699234,14cyclictest4019-21ssh22:55:130
2388699232,18cyclictest6024-21ssh22:20:140
2388699231,16cyclictest7711-21kworker/u8:123:40:120
2388699230,18cyclictest26025-21ssh21:55:150
2392199225,9cyclictest0-21swapper/321:25:123
23921992211,7cyclictest35966irq/58-eth023:55:273
2390799224,10cyclictest9821-21unixbench_multi19:25:272
2390799222,17cyclictest27346irq/40-dwc2_hso23:20:142
2390799222,17cyclictest27346irq/40-dwc2_hso23:20:132
2390799222,17cyclictest12100-21unixbench_multi21:45:292
2390799222,17cyclictest12100-21unixbench_multi21:45:292
2390799222,10cyclictest8443-21wc21:05:292
23907992216,4cyclictest17153-21ssh22:25:202
2390799221,11cyclictest5372-21sh23:50:142
2390799221,11cyclictest23848-21grep00:00:182
23907992210,9cyclictest21765-21kthreadcore22:45:222
2389799223,16cyclictest31560-21ssh00:20:331
2389799222,14cyclictest17632-21sh00:15:111
23897992213,6cyclictest3756-21ssh22:55:021
2389799221,18cyclictest11659-21cut22:05:161
2389799221,15cyclictest27550-21ssh22:30:261
2388699221,14cyclictest1963-21ssh22:15:340
2392199212,12cyclictest35966irq/58-eth022:50:133
2392199212,12cyclictest0-21swapper/322:05:133
2392199212,11cyclictest35966irq/58-eth020:35:133
2392199211,2cyclictest0-21swapper/320:45:123
2390799215,8cyclictest8298-21latency_hist20:35:012
2390799215,13cyclictest13448-21hwlatdetect20:40:172
2390799213,14cyclictest32234-21sendmail21:00:012
2390799211,6cyclictest9563-21df22:40:152
2390799211,16cyclictest31411-21cut22:15:212
2390799211,11cyclictest32629-21ps00:05:192
2390799211,11cyclictest15414-21ntpq20:05:252
2390799211,11cyclictest13640-21fschecks_count21:30:162
2390799211,10cyclictest5720-21cat19:55:222
2390799211,10cyclictest30427-21cut00:20:282
2390799211,10cyclictest18669-21cut00:15:192
2389799213,16cyclictest5369-21ssh23:50:141
2389799213,13cyclictest32268-21ssh00:05:161
23897992114,5cyclictest21-21ksoftirqd/120:40:131
23897992114,3cyclictest21-21ksoftirqd/120:15:121
23897992113,5cyclictest28788-21latency23:25:221
23897992113,5cyclictest28788-21latency23:25:221
23897992112,6cyclictest31834-21ssh21:40:161
23897992112,6cyclictest30912-21ssh23:10:111
23897992112,6cyclictest13041-21ssh23:00:131
2388699219,3cyclictest9398-21diskmemload23:10:200
2388699211,6cyclictest26496-21usage_Anybus23:05:290
2388699211,5cyclictest9398-21diskmemload23:45:280
2388699211,14cyclictest23923-21ls22:10:210
2392199203,11cyclictest0-21swapper/322:10:093
2392199202,6cyclictest0-21swapper/300:25:143
23921992012,6cyclictest0-21swapper/322:20:023
23921992012,5cyclictest0-21swapper/300:22:453
23921992011,6cyclictest35966irq/58-eth021:10:153
2390799201,16cyclictest30130-21grep19:50:182
2390799201,15cyclictest27190-21kthreadcore19:45:212
2390799201,15cyclictest11527-21kthreadcore19:30:202
2390799201,10cyclictest31381-21fschecks_time20:25:162
2390799201,10cyclictest24245-21tune2fs19:10:162
2390799201,10cyclictest10170-21unixbench_singl22:20:312
2389799202,14cyclictest20437-21ssh22:10:111
2389799201,16cyclictest21156-21ssh00:35:141
2389799201,16cyclictest21156-21ssh00:35:131
23897992011,6cyclictest18787-21ssh22:25:271
23897992011,6cyclictest15979-21cron23:20:011
23897992011,6cyclictest14461-21ssh23:55:151
2389799200,8cyclictest131rcu_preempt23:20:141
2389799200,8cyclictest131rcu_preempt23:20:141
2389799200,16cyclictest26523-21cut20:20:171
2388699202,12cyclictest0-21swapper/019:10:130
23886992011,7cyclictest27464-21apt-get22:50:140
23886992011,7cyclictest1797-21cut22:35:200
2388699201,12cyclictest10821-21rm23:35:120
2392199193,10cyclictest0-21swapper/321:50:133
23921991911,5cyclictest35966irq/58-eth022:35:143
2392199191,13cyclictest0-21swapper/319:15:273
23921991910,7cyclictest0-21swapper/322:29:583
23897991913,4cyclictest3210-21cpu00:25:131
23897991913,3cyclictest0-21swapper/121:10:151
2389799191,15cyclictest22017-21rm23:05:131
2389799191,13cyclictest23268-21ssh00:00:121
23897991910,6cyclictest9048-21ssh21:25:311
2388699199,8cyclictest10800-21apt-get19:30:140
2388699193,3cyclictest131rcu_preempt19:35:140
2388699192,9cyclictest30100-21timerandwakeup20:20:280
2388699191,15cyclictest29976-21apt-get19:50:140
2388699191,13cyclictest8773-21ssh00:10:170
23886991910,7cyclictest27188-21date22:50:030
23886991910,6cyclictest5539-21ssh21:25:190
2392199188,8cyclictest0-21swapper/321:23:583
2392199188,3cyclictest0-21swapper/300:34:513
2392199188,3cyclictest0-21swapper/300:34:513
2392199186,8cyclictest35966irq/58-eth023:10:123
2392199183,8cyclictest36066irq/59-eth021:55:153
2392199182,9cyclictest0-21swapper/321:45:143
2392199182,9cyclictest0-21swapper/321:45:133
2392199182,11cyclictest0-21swapper/323:25:143
2392199182,11cyclictest0-21swapper/323:25:133
2392199182,10cyclictest0-21swapper/321:00:143
2392199182,10cyclictest0-21swapper/320:50:153
2392199182,10cyclictest0-21swapper/320:25:143
2392199182,10cyclictest0-21swapper/319:55:053
2392199181,3cyclictest0-21swapper/322:20:143
23921991811,4cyclictest35966irq/58-eth021:15:133
2392199181,13cyclictest0-21swapper/319:30:283
2392199181,11cyclictest0-21swapper/322:00:263
2390799180,8cyclictest23955-21latency_hist19:10:022
2389799183,13cyclictest8988-21kthreadcore19:25:201
2389799183,12cyclictest19394-21kthreadcore21:50:221
2389799182,11cyclictest0-21swapper/120:05:141
2389799181,9cyclictest13143-21kthreadcore23:35:221
23897991812,4cyclictest14208-21nfs_client19:30:231
23897991811,5cyclictest4151-21ssh22:55:141
2389799181,14cyclictest18857-21ssh21:15:151
23897991810,6cyclictest9637-21ssh22:20:251
23897991810,6cyclictest7444-21wc20:00:141
23897991810,5cyclictest986-21grep22:35:181
23897991810,5cyclictest14728-21ssh00:30:221
23897991810,5cyclictest14728-21ssh00:30:221
2388699183,4cyclictest12-21ksoftirqd/022:05:240
2388699181,9cyclictest4762-21sendmail_mailqu19:20:270
23886991812,4cyclictest0-21swapper/020:45:290
23886991812,4cyclictest0-21swapper/020:35:180
23886991811,5cyclictest0-21swapper/021:10:250
2388699181,14cyclictest31789-21ls21:20:290
2388699181,13cyclictest3209-21ssh00:25:130
2388699181,11cyclictest0-21swapper/000:35:540
2388699181,11cyclictest0-21swapper/000:35:530
2388699181,10cyclictest29014-21uniq19:15:150
23886991810,6cyclictest26189-21ntpq21:35:240
2392199179,6cyclictest0-21swapper/322:33:573
2392199179,6cyclictest0-21swapper/300:39:393
2392199179,6cyclictest0-21swapper/300:39:393
2392199179,6cyclictest0-21swapper/300:00:473
2392199179,3cyclictest0-21swapper/322:44:083
2392199172,10cyclictest0-21swapper/320:20:143
2392199172,10cyclictest0-21swapper/320:05:143
2392199172,10cyclictest0-21swapper/320:00:123
2392199172,10cyclictest0-21swapper/319:50:153
2392199171,12cyclictest0-21swapper/319:20:233
2389799179,6cyclictest31452-21ssh21:56:211
2389799179,5cyclictest23156-21irqcore20:50:181
2389799179,5cyclictest1852-21ssh23:30:131
2389799178,7cyclictest30057-21df19:50:161
2389799178,6cyclictest31570-21irqcore20:25:181
2389799176,8cyclictest614-21dbus-daemon19:12:451
2389799174,4cyclictest131rcu_preempt19:45:011
2389799171,7cyclictest8470-21sed21:45:171
2389799171,7cyclictest8470-21sed21:45:171
2389799171,6cyclictest23978-21/usr/sbin/munin23:40:281
23897991713,2cyclictest536-21usb-storage21:35:101
2389799171,14cyclictest9715-21kthreadcore20:35:191
2389799171,10cyclictest2554-21cut19:55:151
23897991710,5cyclictest9401-21cron21:10:011
2388699179,6cyclictest16116-21cat20:05:300
2388699179,5cyclictest12168-21/usr/sbin/munin00:30:270
2388699179,5cyclictest12168-21/usr/sbin/munin00:30:260
2388699178,6cyclictest8112-21kthreadcore23:50:230
2388699177,8cyclictest21601-21apt-get20:15:140
2388699177,7cyclictest24181-21perl22:30:160
2388699177,3cyclictest131rcu_preempt19:50:000
2388699175,9cyclictest6971-21cron20:00:000
2388699171,8cyclictest31279-21cpuspeed20:25:150
2388699171,7cyclictest10580-21cut19:30:010
2388699171,4cyclictest9398-21diskmemload21:43:170
23886991711,4cyclictest0-21swapper/023:30:200
2388699171,13cyclictest32460-21apt-get21:00:130
2388699171,13cyclictest22840-21apt-get20:50:130
2388699171,12cyclictest9460-21ssh22:40:140
2392199168,6cyclictest0-21swapper/323:44:053
2389799169,4cyclictest16359-21kthreadcore19:35:201
2389799167,6cyclictest18032-21apt-get20:45:111
2389799164,4cyclictest131rcu_preempt20:10:141
2389799161,12cyclictest4858-21kthreadcore20:30:191
2389799161,12cyclictest28924-21diskstats19:15:141
2389799161,11cyclictest29753-21grep19:50:011
2389799161,10cyclictest32370-21wc21:00:021
2388699169,5cyclictest13788-21cut20:40:190
2388699168,6cyclictest31904-21cat20:55:300
2388699161,8cyclictest8233-21ntp_192.168.11421:05:260
2388699161,12cyclictest13024-21apt-get23:00:110
2388699161,11cyclictest13492-21sh21:30:120
23886991610,4cyclictest21201-21usage_Anybus21:50:310
2392199152,10cyclictest0-21swapper/319:45:143
2392199151,7cyclictest0-21swapper/320:35:013
2392199151,11cyclictest0-21swapper/320:15:173
2389799157,5cyclictest0-21swapper/119:20:171
2389799153,9cyclictest324-21grep21:00:191
2388699158,5cyclictest17037-21basename20:10:160
2388699157,6cyclictest4221-21irqstats20:30:170
2388699151,6cyclictest24209-21unixbench_multi19:40:280
2392199149,3cyclictest0-21swapper/320:10:523
2392199142,10cyclictest0-21swapper/321:05:143
2392199141,8cyclictest0-21swapper/319:40:113
2392199141,10cyclictest0-21swapper/320:55:283
2392199140,10cyclictest35966irq/58-eth020:40:143
2388699141,10cyclictest23937-21latency_hist19:10:020
2392199132,9cyclictest0-21swapper/319:25:123
2389799132,8cyclictest23953-21latency_hist19:10:021
2392199121,9cyclictest0-21swapper/319:35:173
2392199121,8cyclictest0-21swapper/319:15:023
2392199111,7cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional