You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 00:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Apr 23, 2026 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
588999598,28cyclictest35966irq/58-eth011:55:113
588999531,23cyclictest35966irq/58-eth012:20:113
587399513,36cyclictest1566-21ssh10:45:141
588999501,29cyclictest35966irq/58-eth008:25:123
5873994918,24cyclictest21-21ksoftirqd/111:55:131
586499454,10cyclictest131rcu_preempt07:15:130
586499453,7cyclictest131rcu_preempt09:05:120
5864994517,3cyclictest131rcu_preempt09:10:090
586499451,39cyclictest32488-21ssh11:20:110
588999444,24cyclictest35966irq/58-eth010:40:113
586499441,34cyclictest8398-21ssh09:55:120
588999434,23cyclictest35966irq/58-eth010:30:123
588999432,26cyclictest35966irq/58-eth010:45:143
587399422,22cyclictest28184-21ssh09:30:131
586499423,32cyclictest23455-21ssh11:15:120
588199411,20cyclictest3689-21ssh11:40:122
586499403,32cyclictest10875-21ssh12:20:120
588999392,24cyclictest35966irq/58-eth012:10:113
586499391,33cyclictest16260-21ssh12:05:110
586499386,28cyclictest4191-21kworker/u8:212:10:120
587399373,8cyclictest131rcu_preempt10:50:121
588199361,18cyclictest28691-21ssh10:40:252
586499365,7cyclictest131rcu_preempt10:15:120
588999351,22cyclictest35966irq/58-eth011:30:113
586499344,22cyclictest12-21ksoftirqd/011:45:140
586499343,19cyclictest12-21ksoftirqd/008:35:120
586499341,28cyclictest13775-21ssh09:40:130
588999335,19cyclictest35966irq/58-eth012:00:123
588199331,22cyclictest5381-21sh11:05:132
586499338,18cyclictest12-21ksoftirqd/007:55:130
588199321,27cyclictest10870-21ssh12:20:112
587399322,21cyclictest22870-21ssh09:25:261
5889993123,5cyclictest33-21ksoftirqd/312:30:133
588999311,17cyclictest35966irq/58-eth011:20:123
588199310,14cyclictest1-21systemd07:20:122
587399314,18cyclictest30304-21ssh10:25:121
588999303,13cyclictest0-21swapper/312:15:143
588999301,17cyclictest35966irq/58-eth010:00:123
588199302,11cyclictest21130-21ntp_states10:00:262
588199301,21cyclictest7249-21ssh12:00:122
586499304,23cyclictest17476-21ssh10:00:140
588999294,14cyclictest35966irq/58-eth009:55:123
588999291,18cyclictest35966irq/58-eth010:15:133
5889992911,8cyclictest0-21swapper/310:10:113
588199292,11cyclictest16545-21ssh09:59:252
587399293,20cyclictest22857-21ssh09:45:141
587399291,20cyclictest10161-21ssh09:20:151
586499292,20cyclictest1546-21ssh10:45:140
588999285,12cyclictest35966irq/58-eth011:00:133
5889992811,12cyclictest35966irq/58-eth011:45:143
588199282,20cyclictest1386-21sh10:08:242
588199281,24cyclictest930-21snmpd07:15:142
588199281,18cyclictest21203-21ssh10:20:132
587399282,20cyclictest31566-21ssh11:19:231
587399282,20cyclictest21-21ksoftirqd/109:35:121
587399281,19cyclictest3071-21ssh10:10:041
587399281,17cyclictest5387-21sh12:35:111
586499284,18cyclictest19607-21ssh10:55:130
586499283,19cyclictest5400-21ssh11:05:140
588999273,16cyclictest0-21swapper/310:20:123
588999272,18cyclictest0-21swapper/311:19:243
588999271,4cyclictest0-21swapper/311:25:133
588199274,13cyclictest27346irq/40-dwc2_hso08:20:132
588199273,17cyclictest27346irq/40-dwc2_hso08:40:132
588199272,20cyclictest0-21swapper/207:40:142
588199272,18cyclictest27921-21ssh12:29:242
588199272,14cyclictest12710-21ssh11:45:132
588199271,4cyclictest0-21swapper/209:15:132
588199271,11cyclictest0-21swapper/210:45:152
587399271,20cyclictest28677-21ssh11:00:131
5873992710,11cyclictest0-21swapper/109:15:131
586499275,15cyclictest630-21Async09:36:310
586499273,15cyclictest12-21ksoftirqd/009:45:120
586499272,19cyclictest21674-21ssh11:50:130
588999261,11cyclictest35966irq/58-eth010:55:123
5889992611,10cyclictest0-21swapper/310:08:253
588199264,14cyclictest27346irq/40-dwc2_hso08:50:112
588199263,14cyclictest27346irq/40-dwc2_hso07:50:122
588199262,9cyclictest0-21swapper/211:00:132
588199262,17cyclictest27346irq/40-dwc2_hso08:35:102
587399264,15cyclictest25005-21ssh10:40:121
587399263,15cyclictest28838-21ssh12:30:141
587399262,19cyclictest1387-21ssh10:08:251
586499269,12cyclictest12-21ksoftirqd/010:05:540
588999253,15cyclictest0-21swapper/312:29:243
588199254,16cyclictest27346irq/40-dwc2_hso08:15:132
588199252,17cyclictest2828-21ssh12:32:242
588199252,14cyclictest27346irq/40-dwc2_hso09:05:122
5881992514,9cyclictest27-21ksoftirqd/210:15:112
5881992510,7cyclictest0-21swapper/208:05:172
5881992510,7cyclictest0-21swapper/208:05:162
587399252,19cyclictest24523-21sh09:10:111
5873992512,9cyclictest5855-21cyclictest07:40:131
586499258,10cyclictest0-21swapper/012:00:120
586499251,18cyclictest32000-21ssh11:35:330
588199243,9cyclictest0-21swapper/208:10:122
588199242,20cyclictest5356-21netstat12:15:232
588199242,19cyclictest0-21swapper/207:10:122
588199242,12cyclictest25255-21ssh12:10:122
588199241,7cyclictest24143-21diskmemload11:10:132
5881992414,8cyclictest0-21swapper/207:30:282
586499242,18cyclictest32521-21ssh12:30:250
586499242,16cyclictest22782-21ssh09:25:240
588999237,10cyclictest35966irq/58-eth007:15:133
588999233,13cyclictest0-21swapper/311:06:243
588199238,12cyclictest20374-21sh10:55:192
588199233,16cyclictest0-21swapper/211:50:132
588199232,6cyclictest0-21swapper/210:39:252
588199232,16cyclictest0-21swapper/211:19:242
588199232,16cyclictest0-21swapper/210:51:242
588199232,14cyclictest27346irq/40-dwc2_hso11:20:132
5881992311,9cyclictest29347-21ssh09:10:292
587399234,10cyclictest0-21swapper/107:20:121
586499233,4cyclictest131rcu_preempt08:50:110
586499232,17cyclictest19919-21ssh12:25:140
588199225,11cyclictest0-21swapper/207:45:102
5881992211,5cyclictest15005-21ssh11:26:492
588199221,10cyclictest32169-21timerwakeupswit09:30:292
587399227,11cyclictest1-21systemd07:25:141
587399223,10cyclictest131rcu_preempt11:25:121
587399221,17cyclictest21657-21ssh11:30:241
5873992210,8cyclictest21-21ksoftirqd/111:10:271
586499224,9cyclictest131rcu_preempt10:35:230
586499222,6cyclictest131rcu_preempt09:50:140
586499221,18cyclictest2731-21cut07:40:160
586499221,17cyclictest4716-21ntp_192.168.11409:15:250
5889992113,5cyclictest35966irq/58-eth010:50:113
588999211,2cyclictest0-21swapper/310:25:123
588999211,11cyclictest35966irq/58-eth011:40:123
588999211,11cyclictest35966irq/58-eth007:35:133
588199218,10cyclictest5102-21irqcore09:35:182
588199217,3cyclictest18306-21kthreadcore12:05:212
588199212,6cyclictest0-21swapper/209:40:112
588199211,17cyclictest21891-21grep11:30:262
588199211,12cyclictest1070-21kthreadcore09:50:212
587399215,11cyclictest21-21ksoftirqd/111:35:121
587399211,16cyclictest13763-21apt-get09:40:121
586499215,10cyclictest12-21ksoftirqd/008:15:150
586499214,15cyclictest1762-21ssh12:15:130
586499213,16cyclictest3695-21ssh11:40:130
586499212,14cyclictest28314-21diskstats09:30:160
586499211,11cyclictest19342-21apt09:05:020
588999209,8cyclictest0-21swapper/309:26:123
588999209,6cyclictest0-21swapper/312:36:553
588999203,13cyclictest0-21swapper/309:30:133
588999202,11cyclictest0-21swapper/308:05:133
588999202,11cyclictest0-21swapper/308:05:133
588999202,10cyclictest35966irq/58-eth009:40:133
588199209,9cyclictest24457-21ssh09:26:012
588199209,9cyclictest0-21swapper/210:25:252
588199209,4cyclictest0-21swapper/209:45:172
588199207,10cyclictest0-21swapper/207:25:212
588199206,11cyclictest20102-21kthreadcore08:30:202
588199201,9cyclictest30756-21ntpq11:35:252
588199201,9cyclictest22901-21kthreadcore08:00:192
588199201,16cyclictest3485-21wc10:10:162
588199201,16cyclictest10815-21irqrtprio09:20:182
588199201,15cyclictest32698-21fschecks_count08:45:162
588199201,15cyclictest16265-21kthreadcore09:00:192
588199201,10cyclictest31695-21kthreadcore07:35:202
588199201,10cyclictest2999-21ssh11:55:312
5881992010,7cyclictest7072-21fschecks_count10:30:152
587399209,7cyclictest26041-21tail07:30:201
587399203,11cyclictest0-21swapper/107:45:101
587399202,13cyclictest7364-21ssh12:00:141
587399201,9cyclictest22816-21cat09:05:211
587399201,3cyclictest12179-21apt-get10:15:121
587399201,15cyclictest3581-21ssh11:20:231
587399201,10cyclictest9922-21kthreadcore10:30:211
586499207,3cyclictest131rcu_preempt10:50:280
586499202,14cyclictest21194-21ssh10:20:130
586499201,9cyclictest9839-21sshd09:35:490
586499201,16cyclictest13597-21grep08:25:160
586499201,14cyclictest844-21munin-node08:10:140
588999198,8cyclictest0-21swapper/311:37:183
588999198,6cyclictest0-21swapper/309:13:173
588999192,11cyclictest0-21swapper/308:50:143
588999192,11cyclictest0-21swapper/307:30:143
588999192,11cyclictest0-21swapper/307:20:123
5889991911,6cyclictest0-21swapper/309:17:333
588199191,9cyclictest6494-21ssh12:35:192
588199191,9cyclictest19655-21kthreadcore07:55:212
588199191,9cyclictest15568-21kthreadcore08:25:202
588199191,9cyclictest12462-21kthreadcore08:55:202
587399195,3cyclictest131rcu_preempt12:25:151
587399192,4cyclictest53646usb-storage08:50:051
587399192,12cyclictest4191-21kworker/u8:212:20:041
587399191,9cyclictest0-21swapper/112:08:451
587399191,15cyclictest21657-21apt-get11:50:121
587399190,6cyclictest13146-21munin-plugin-st08:25:001
587399190,14cyclictest5778-21timerwakeupswit12:15:291
587399190,13cyclictest17971-21latency_hist08:30:021
586499199,8cyclictest13142-21cron08:25:010
586499192,14cyclictest6915-21ssh10:30:120
586499192,12cyclictest18157-21ssh11:30:130
586499192,12cyclictest13907-21users11:25:310
586499192,11cyclictest151rcuc/011:10:130
586499191,9cyclictest0-21swapper/011:55:040
586499191,15cyclictest10151-21ssh09:20:140
588999189,6cyclictest35966irq/58-eth011:50:123
588999183,11cyclictest0-21swapper/310:39:253
588999182,11cyclictest0-21swapper/308:15:153
588999182,10cyclictest0-21swapper/309:05:123
588999182,10cyclictest0-21swapper/308:45:143
588999182,10cyclictest0-21swapper/308:30:123
588999182,10cyclictest0-21swapper/308:20:133
588999182,10cyclictest0-21swapper/308:10:133
588999182,10cyclictest0-21swapper/308:00:133
588999181,12cyclictest35966irq/58-eth011:10:133
588999181,10cyclictest0-21swapper/308:35:133
588999181,10cyclictest0-21swapper/307:45:023
588199181,13cyclictest6033-21latency_hist07:10:022
587399189,7cyclictest9511-21ntpq07:10:231
587399189,5cyclictest5359-21acpi11:05:111
587399188,7cyclictest18871-21tr10:35:211
587399188,7cyclictest13299-21ntpq08:55:241
587399185,7cyclictest5855-21cyclictest08:30:121
587399182,11cyclictest24894-21ssh10:56:011
587399181,4cyclictest0-21swapper/111:44:361
5873991810,6cyclictest22027-21cat10:00:291
586499183,5cyclictest131rcu_preempt11:00:160
586499182,4cyclictest12-21ksoftirqd/008:40:120
5864991814,3cyclictest536-21usb-storage10:10:110
586499181,2cyclictest131rcu_preempt10:40:120
5864991811,4cyclictest8412-21cut12:35:220
586499181,13cyclictest30866-21irqrtprio10:25:180
588999179,6cyclictest0-21swapper/309:45:383
588999179,5cyclictest35966irq/58-eth009:35:113
588999172,10cyclictest0-21swapper/307:55:143
588999172,10cyclictest0-21swapper/307:45:113
588999172,10cyclictest0-21swapper/307:25:133
588999171,9cyclictest35966irq/58-eth009:50:133
5889991711,3cyclictest0-21swapper/309:20:183
587399177,7cyclictest21-21ksoftirqd/108:00:211
587399176,8cyclictest21-21ksoftirqd/110:20:131
587399172,13cyclictest31551-21fschecks_time08:10:151
587399172,11cyclictest8497-21df09:55:141
587399172,10cyclictest0-21swapper/109:00:141
587399171,8cyclictest15953-21unixbench_multi07:50:271
5873991711,4cyclictest12689-21/usr/sbin/munin11:45:271
587399171,13cyclictest31919-21sh09:50:151
587399171,12cyclictest27805-21apt-get08:40:121
5873991710,5cyclictest32742-1kworker/u9:112:11:171
587399170,15cyclictest4912-21kthreadcore08:15:211
586499178,7cyclictest21998-21grep08:00:160
586499178,7cyclictest15785-21diskstats07:20:150
586499178,6cyclictest27468-21kthreadcore08:05:190
586499178,6cyclictest27468-21kthreadcore08:05:180
586499173,12cyclictest32590-21/usr/sbin/munin08:45:140
586499173,11cyclictest10390-21tr07:45:180
586499172,12cyclictest21994-21sed08:30:280
586499172,11cyclictest710-21cron09:00:010
5864991710,5cyclictest27093-21cut07:30:220
588999162,9cyclictest0-21swapper/309:00:053
588999162,10cyclictest0-21swapper/312:05:143
588999161,12cyclictest0-21swapper/308:55:263
587399167,6cyclictest27564-21munin-run08:40:021
587399162,9cyclictest0-21swapper/107:35:141
587399162,11cyclictest32760-21fschecks_time08:45:151
587399161,12cyclictest14045-21tr07:15:231
587399161,10cyclictest28877-21kthreadcore08:05:201
587399161,10cyclictest28877-21kthreadcore08:05:201
586499169,4cyclictest0-21swapper/007:35:230
586499167,7cyclictest24045-21tr07:25:260
586499163,10cyclictest14624-21kthreadcore07:50:210
588999152,10cyclictest0-21swapper/308:40:133
587399151,11cyclictest17096-21fschecks_time07:55:161
586499151,11cyclictest5956-21latency_hist07:10:000
587399141,10cyclictest5942-21latency_hist07:10:011
588999138,3cyclictest0-21swapper/307:13:063
588999133,8cyclictest0-21swapper/307:50:163
588999101,6cyclictest0-21swapper/307:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional