You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-09 - 19:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Aug 09, 2026 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
210579913544,68cyclictest18028-21ssh10:50:200
210579913526,86cyclictest0-21swapper/010:25:260
210579913125,84cyclictest0-21swapper/011:12:350
21057991253,71cyclictest0-21swapper/010:00:320
2360921190,4sleep022585-21/usr/sbin/munin11:35:310
21061991173,74cyclictest17687-21ssh10:50:063
21061991153,73cyclictest847-21zcat09:10:293
21057991132,34cyclictest755-21ssh09:10:280
210599911117,52cyclictest0-21swapper/212:00:072
210599911117,52cyclictest0-21swapper/212:00:072
21061991103,85cyclictest9996-21grep10:00:333
21061991063,68cyclictest14211-21awk10:05:313
21061991063,53cyclictest30256-21/usr/sbin/munin10:25:253
21061991052,62cyclictest30838-21apt-get11:45:153
21061991052,62cyclictest30838-21apt-get11:45:153
210619910512,56cyclictest7850irq/56-dwc_otg_12:35:043
210589910516,59cyclictest0-21swapper/112:00:081
210589910516,59cyclictest0-21swapper/112:00:081
2016521050,6sleep02105799cyclictest08:35:280
21061991042,73cyclictest0-21swapper/310:55:053
210599910441,43cyclictest7950irq/56-dwc_otg_10:25:232
210579910434,9cyclictest0-21swapper/010:35:270
21057991035,89cyclictest0-21swapper/012:35:030
210579910346,21cyclictest21615-21grep10:15:210
210579910126,6cyclictest0-21swapper/009:25:300
21057991012,28cyclictest7950irq/56-dwc_otg_09:50:150
286442990,2sleep20-21swapper/209:00:092
2105799984,9cyclictest5368-21memory09:55:240
2105799976,27cyclictest0-21swapper/011:45:140
2105799976,27cyclictest0-21swapper/011:45:140
2105799963,30cyclictest0-21swapper/010:05:330
21057999425,19cyclictest14205-21ssh10:45:240
21059999138,16cyclictest14284-21ssh10:05:332
2106199905,60cyclictest7850irq/56-dwc_otg_12:05:263
21059999040,28cyclictest7850irq/56-dwc_otg_10:00:332
21057999027,8cyclictest7750irq/56-dwc_otg12:00:150
21057999027,8cyclictest7750irq/56-dwc_otg12:00:150
21059998938,25cyclictest30851-21ssh11:45:132
21059998938,25cyclictest30851-21ssh11:45:132
21057998834,8cyclictest7950irq/56-dwc_otg_12:05:270
21057998833,36cyclictest0-21swapper/010:30:260
21057998828,5cyclictest0-21swapper/009:05:160
21057998826,6cyclictest0-21swapper/010:20:210
2105799882,30cyclictest28947-21ssh09:45:170
21059998735,15cyclictest17693-21ssh10:50:082
2105899864,62cyclictest7850irq/56-dwc_otg_09:10:281
2106199853,76cyclictest12745-21fschecks_time09:25:233
2105899852,58cyclictest7750irq/56-dwc_otg10:00:321
2106199842,74cyclictest25908-21ssh10:20:233
21059998439,23cyclictest792-21ssh09:10:282
2106199834,59cyclictest7850irq/56-dwc_otg_12:00:143
2106199834,59cyclictest7850irq/56-dwc_otg_12:00:143
2106199833,73cyclictest2042-21grep10:30:293
2105899831,49cyclictest18028-21ssh10:50:201
2106199823,72cyclictest6914-21ssh11:55:173
2106199823,72cyclictest23357-21ntpq11:35:263
2106199823,71cyclictest28924-21apt-get09:45:173
2106199823,71cyclictest21613-21/usr/sbin/munin10:15:213
2106199822,61cyclictest14201-21memory10:45:243
21059998251,24cyclictest21719-21ssh10:15:232
2106199814,57cyclictest7850irq/56-dwc_otg_11:12:353
2106199813,71cyclictest4717-21/usr/sbin/munin09:55:263
21059998140,23cyclictest25897-21ssh10:20:232
2105799813,27cyclictest0-21swapper/011:55:180
2106199803,56cyclictest7950irq/56-dwc_otg_10:35:173
2105799805,17cyclictest7850irq/56-dwc_otg_09:35:200
2105799805,17cyclictest7850irq/56-dwc_otg_09:35:200
2106199783,68cyclictest602-21ssh09:50:143
2105899788,43cyclictest6980-21ssh11:55:181
21058997839,15cyclictest60-21kswapd007:20:091
2105899783,53cyclictest0-21swapper/109:25:221
21058997811,43cyclictest7750irq/56-dwc_otg12:05:261
65762770,8sleep0101ktimersoftd/008:00:020
21058997733,22cyclictest29470-21/usr/sbin/munin10:25:121
2106199767,49cyclictest7950irq/56-dwc_otg_08:35:173
21059997633,21cyclictest4270-21ssh11:12:362
21058997615,40cyclictest4277-21ssh11:12:371
21058997518,39cyclictest7750irq/56-dwc_otg10:35:291
2105899748,36cyclictest7850irq/56-dwc_otg_09:50:161
2105899746,41cyclictest7850irq/56-dwc_otg_12:35:031
2105899745,32cyclictest0-21swapper/111:45:141
2105899745,32cyclictest0-21swapper/111:45:141
59752730,2sleep20-21swapper/212:33:102
2106199721,6cyclictest32336-21/usr/sbin/munin07:40:213
21059997244,23cyclictest13175-21ssh09:25:282
21059997213,35cyclictest7750irq/56-dwc_otg11:55:192
21058997235,17cyclictest3172-21/usr/sbin/munin07:50:301
21058997235,17cyclictest3172-21/usr/sbin/munin07:50:301
21059997145,21cyclictest5366-21ssh10:35:152
21059997136,13cyclictest28270-21latency_hist07:30:032
2105899713,44cyclictest25958-21latency10:20:241
21058997130,27cyclictest7850irq/56-dwc_otg_10:30:281
21058997130,22cyclictest26171-21ssh09:40:391
21058997118,32cyclictest0-21swapper/110:55:051
21058997112,33cyclictest7750irq/56-dwc_otg10:05:321
2105899704,45cyclictest7750irq/56-dwc_otg10:15:211
21058997031,21cyclictest11543-21rm10:41:571
301642690,4sleep030139-21apt-get11:05:150
301642690,4sleep030139-21apt-get11:05:150
269742690,2sleep30-21swapper/311:00:273
21059996931,21cyclictest31846-21latency_hist07:40:022
21058996935,10cyclictest5959-21ssh11:15:061
21058996935,10cyclictest5959-21ssh11:15:061
21059996842,21cyclictest28938-21ssh09:45:152
21059996811,32cyclictest15794-21ntp_states12:05:262
2105899687,33cyclictest7850irq/56-dwc_otg_10:45:231
21058996836,12cyclictest20169-21/usr/sbin/munin08:35:281
21058996812,8cyclictest7750irq/56-dwc_otg11:35:281
2106199676,45cyclictest7950irq/56-dwc_otg_08:40:233
2106199676,45cyclictest7950irq/56-dwc_otg_08:40:233
2106199672,60cyclictest7237-21rm12:35:053
21059996730,14cyclictest4281-21unixbench_multi07:50:332
21059996730,14cyclictest4281-21unixbench_multi07:50:332
21059996729,13cyclictest32382-21basename07:40:222
21059996713,34cyclictest14277-21ls10:45:252
21058996729,18cyclictest22331-21unixbench_multi08:40:301
21058996729,18cyclictest22331-21unixbench_multi08:40:301
21059996638,23cyclictest32040-21diskmemload11:35:272
2105899664,46cyclictest0-21swapper/109:45:151
21058996639,8cyclictest10195-21ssh11:20:141
21057996627,17cyclictest1180-21runrttasks10:55:050
21059996538,22cyclictest5444-21ssh09:55:252
2105899653,7cyclictest0-21swapper/109:55:261
21058996528,18cyclictest21470-21awk09:35:261
21058996528,18cyclictest21470-21awk09:35:261
21059996427,20cyclictest18777-21ssh09:32:172
21059996425,10cyclictest0-21swapper/209:05:232
21058996430,20cyclictest12263-21wget08:15:161
21058996428,22cyclictest30213-21ssh12:23:171
2106199631,45cyclictest13027-21ssh10:44:413
2105999634,10cyclictest7750irq/56-dwc_otg09:50:142
21059996329,17cyclictest28669-21sh12:20:302
2105799636,35cyclictest0-21swapper/012:35:050
2106199621,4cyclictest7950irq/56-dwc_otg_09:00:093
2106199621,4cyclictest24783-21/usr/sbin/munin09:40:203
21058996232,18cyclictest31168-21ntpq09:05:271
21058996229,16cyclictest23594-21fschecks_time12:15:201
21058996227,19cyclictest3234-21missed_timers11:50:241
21058996227,16cyclictest32188-21ssh11:06:521
21058996227,16cyclictest32188-21ssh11:06:521
21058996225,22cyclictest30310-21sed09:05:021
2106199617,31cyclictest0-21swapper/312:26:213
2106199612,51cyclictest4278-21grep07:50:333
2106199612,51cyclictest4278-21grep07:50:333
2106199611,5cyclictest5605-21apt-get11:15:043
2106199611,5cyclictest5605-21apt-get11:15:043
21059996132,13cyclictest28428-21apt-config09:00:022
21059996131,12cyclictest22296-21ls08:40:292
21059996131,12cyclictest22296-21ls08:40:292
21059996129,13cyclictest10384-21cat11:20:172
21058996129,18cyclictest31572-21sshd07:37:461
21058996128,22cyclictest2336-21ssh12:29:031
21058996128,16cyclictest20490-21ssh10:14:191
21058996125,25cyclictest11731-21rm09:24:301
2105799611,40cyclictest28641-21fschecks_count07:30:230
2106199606,40cyclictest0-21swapper/311:31:173
21059996040,15cyclictest1984-21ssh10:30:272
21059996031,12cyclictest1587-21df_inode07:45:182
21059996030,15cyclictest3326-21ntpq11:50:262
21059996027,15cyclictest23099-21latency_hist12:15:022
21059996024,11cyclictest26338-21taskset07:24:522
2105899607,40cyclictest7850irq/56-dwc_otg_12:35:051
21058996032,20cyclictest939-21seq07:42:591
2105799606,17cyclictest9-21ksoftirqd/011:25:180
21057996046,5cyclictest1953-21ls07:45:240
2106199595,42cyclictest0-21swapper/309:37:023
2106199595,42cyclictest0-21swapper/309:37:023
21059995933,21cyclictest7242-21ssh12:35:062
21059995928,17cyclictest25523-21rm10:59:472
21059995927,19cyclictest15944-21ssh11:26:052
21059995927,17cyclictest27679-21ssh09:43:252
21059995923,15cyclictest13436-21latency_hist10:45:032
21058995944,12cyclictest21056-21cyclictest07:10:031
21058995932,11cyclictest24225-21runrttasks07:17:141
21058995929,13cyclictest20438-21ssh09:35:031
21058995928,10cyclictest20344-21ssh11:31:351
21058995927,17cyclictest28297-21chrt08:59:561
21058995927,12cyclictest6187-21chrt07:55:401
21058995924,16cyclictest7950irq/56-dwc_otg_11:45:031
21059995830,11cyclictest23791-21ssh12:15:232
21059995827,16cyclictest27826-21ssh11:01:262
21059995826,20cyclictest24495-21seq08:48:402
21058995833,14cyclictest5841-21ssh09:16:181
21058995827,14cyclictest21620-21ssh12:12:391
21058995823,16cyclictest0-21swapper/108:08:491
21058995823,11cyclictest7950irq/56-dwc_otg_07:45:191
2106199577,39cyclictest7750irq/56-dwc_otg10:10:483
2106199572,4cyclictest0-21swapper/311:20:213
2106199571,6cyclictest7950irq/56-dwc_otg_07:45:223
2106199571,46cyclictest17307-21ssh11:28:353
21059995731,15cyclictest19012-21ssh10:11:372
21059995727,16cyclictest5650-21ssh09:15:582
21059995726,13cyclictest21636-21ssh11:33:562
21059995723,12cyclictest8653-21ssh11:17:512
21059995723,12cyclictest8653-21ssh11:17:512
21059995720,25cyclictest0-21swapper/207:19:342
21058995736,7cyclictest6955-21cpu08:00:181
21058995727,12cyclictest27624-21ssh11:01:041
21058995725,20cyclictest26244-21taskset08:51:441
2105799571,29cyclictest24458-21apt-get07:20:090
2106199563,4cyclictest7850irq/56-dwc_otg_08:30:023
2106199562,5cyclictest30459-21munin-node09:05:133
2106199562,41cyclictest0-21swapper/312:12:553
2106199561,4cyclictest7950irq/56-dwc_otg_11:09:483
2106199561,4cyclictest7950irq/56-dwc_otg_11:09:483
2106199561,4cyclictest23824-21ssh12:15:233
21061995612,31cyclictest3467-21/usr/sbin/munin11:50:283
21059995630,17cyclictest11907-21taskset08:14:302
21059995628,19cyclictest32267-21ssh11:07:002
21059995628,19cyclictest32267-21ssh11:07:002
21059995626,12cyclictest30432-21sh11:44:542
21059995625,11cyclictest2365-21ssh12:29:072
21058995629,11cyclictest15752-21ssh11:25:441
2105799563,37cyclictest0-21swapper/007:25:120
21057995611,9cyclictest18757-21unixbench_singl08:30:310
2106199555,4cyclictest7850irq/56-dwc_otg_11:40:163
21059995527,8cyclictest21950-21users09:35:312
21059995527,8cyclictest21950-21users09:35:312
21059995527,12cyclictest9989-21ssh09:21:192
21059995522,19cyclictest20128-21ntp_states08:35:262
21057995530,5cyclictest22233-21grep07:10:320
21057995529,5cyclictest26035-21unixbench-2d08:50:310
2105799551,31cyclictest140-21systemd-journal12:30:020
2106199546,31cyclictest7850irq/56-dwc_otg_12:20:173
2106199542,48cyclictest25036-21fschecks_count07:20:193
2106199541,3cyclictest0-21swapper/309:15:233
21059995427,11cyclictest10073-21chrt08:09:092
21058995427,8cyclictest15355-21taskset08:22:201
2106199532,4cyclictest7950irq/56-dwc_otg_07:20:033
2106199532,47cyclictest26941-21fschecks_time07:25:193
2106199531,5cyclictest7950irq/56-dwc_otg_08:30:233
21059995325,17cyclictest6194-21sshd07:55:452
21059995325,12cyclictest7850irq/56-dwc_otg_08:30:182
21058995325,14cyclictest0-21swapper/108:11:521
21057995333,13cyclictest24853-21cut09:40:180
2106199522,46cyclictest1180-21runrttasks08:50:033
2106199522,45cyclictest21146-21grep07:10:173
2106199521,4cyclictest30210-21apt-get07:35:143
2106199521,47cyclictest1180-21runrttasks08:50:053
21058995227,11cyclictest0-21swapper/108:27:341
21058995224,18cyclictest29764-21runrttasks07:32:381
21058995224,12cyclictest0-21swapper/108:49:001
21057995228,10cyclictest7950irq/56-dwc_otg_10:40:280
2105799522,27cyclictest0-21swapper/007:40:320
21057995220,20cyclictest7950irq/56-dwc_otg_11:30:200
2105799521,6cyclictest15912-21aten_rbpower_po08:25:170
2105799521,30cyclictest7335-21grep11:15:310
2105799521,30cyclictest7335-21grep11:15:310
21059995124,8cyclictest15585-21munin-run08:25:022
21059995123,17cyclictest8040-21seq08:01:182
21059995123,11cyclictest1180-21runrttasks08:16:482
21059995122,11cyclictest9550irq/80-mmc008:50:082
2106199502,10cyclictest7850irq/56-dwc_otg_08:05:163
2106199501,5cyclictest7197-21if_eth008:00:223
2106199501,5cyclictest0-21swapper/309:21:283
2106199501,39cyclictest7950irq/56-dwc_otg_07:35:013
21057995024,5cyclictest28927-21cut09:00:210
2105799501,29cyclictest32040-21diskmemload11:20:270
2105799501,24cyclictest10151-21modprobe08:10:010
2105799501,24cyclictest0-21swapper/008:55:140
117332500,2sleep00-21swapper/009:24:300
2106199498,38cyclictest5516-21sed07:55:223
2106199492,5cyclictest12193-21/usr/sbin/munin08:15:303
2106199492,44cyclictest14517-21grep08:20:213
2106199491,7cyclictest16487-21/usr/sbin/munin09:30:283
21059994938,8cyclictest21089-21apt-get07:10:142
21059994924,8cyclictest0-21swapper/207:30:212
21058994924,13cyclictest0-21swapper/108:30:111
21057994923,15cyclictest7950irq/56-dwc_otg_12:10:190
21057994916,14cyclictest2598350ssh11:00:120
21057994912,12cyclictest23202-21sh12:15:080
2106199482,41cyclictest0-21swapper/309:00:023
2106199471,5cyclictest10722-21fschecks_time08:10:213
21057994726,5cyclictest17313-21df10:10:170
2105799471,36cyclictest0-21swapper/009:16:270
2105799471,15cyclictest32040-21diskmemload09:34:060
21059994631,12cyclictest16414-21/usr/sbin/munin08:25:252
2105799462,27cyclictest0-21swapper/011:42:080
2105799461,7cyclictest12489-21fschecks_time08:15:190
2105799461,35cyclictest9-21ksoftirqd/011:54:380
21058994528,14cyclictest27584-21ls07:25:291
21057994515,12cyclictest0-21swapper/007:50:280
21057994515,12cyclictest0-21swapper/007:50:280
21057994510,17cyclictest0-21swapper/012:20:150
21057994321,13cyclictest0-21swapper/007:40:020
2105799421,24cyclictest22344-21sed08:40:300
2105799421,24cyclictest22344-21sed08:40:300
2105799421,23cyclictest0-21swapper/008:50:020
2105799411,24cyclictest0-21swapper/008:13:450
2105799411,21cyclictest14770-21ps08:20:260
2105799403,22cyclictest9-21ksoftirqd/008:00:310
2105799391,36cyclictest0-21swapper/007:17:250
192502374,24sleep219304-21aten_rbpower_en07:05:152
198752320,2sleep30-21swapper/307:05:233
205102293,16sleep10-21swapper/107:06:321
196662242,12sleep00-21swapper/007:05:210
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional