You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 21:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Jun 28, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1135423260,8sleep123-21ksoftirqd/107:05:311
12477991136,81cyclictest0-21swapper/010:35:230
12477991136,81cyclictest0-21swapper/010:35:230
21132980,2sleep00-21swapper/009:22:310
1247799953,72cyclictest0-21swapper/009:25:340
212422910,2sleep00-21swapper/009:45:300
139882830,2sleep30-21swapper/312:13:403
1247899832,59cyclictest12001-21sed12:10:361
1247799834,14cyclictest0-21swapper/008:55:240
177272820,2sleep10-21swapper/111:00:111
12477997959,11cyclictest0-21swapper/010:45:220
12477997739,8cyclictest7950irq/56-dwc_otg_12:35:210
12477997739,8cyclictest7950irq/56-dwc_otg_12:35:210
1247899763,66cyclictest4735-21tr09:25:331
1247799753,49cyclictest17337-21grep08:50:350
1247799753,49cyclictest17337-21grep08:50:350
1248099747,14cyclictest0-21swapper/308:20:223
12479997332,20cyclictest8050irq/56-dwc_otg_09:25:332
222422720,2sleep20-21swapper/212:23:412
1247899722,47cyclictest8050irq/56-dwc_otg_10:10:211
140412700,5sleep11247899cyclictest11:35:091
140412700,5sleep11247899cyclictest11:35:091
1247999704,24cyclictest30256-21ssh11:15:222
12477997036,14cyclictest0-21swapper/007:15:210
1247799696,60cyclictest21816-21grep07:35:260
1247799692,58cyclictest7230-21basename08:25:240
1247799692,58cyclictest7230-21basename08:25:240
12477996910,44cyclictest101ktimersoftd/011:35:100
12477996910,44cyclictest101ktimersoftd/011:35:100
12477996910,39cyclictest30312-21grep11:15:230
1247899685,47cyclictest24729-21ssh09:10:371
1247799682,47cyclictest0-21swapper/009:45:090
292132670,6sleep11247899cyclictest09:15:401
1247899671,56cyclictest14749-21/usr/sbin/munin08:45:281
1247799679,35cyclictest0-21swapper/008:36:390
12477996710,47cyclictest24787-21aten_rbpower_cu09:50:220
12480996615,25cyclictest7950irq/56-dwc_otg_09:25:333
1247899662,35cyclictest0-21swapper/112:36:161
1247899662,35cyclictest0-21swapper/112:36:161
1247799668,40cyclictest13299-21switchtime09:35:350
1247799662,49cyclictest0-21swapper/011:35:160
12478996510,42cyclictest29667-21sh12:31:571
1247799654,47cyclictest0-21swapper/010:55:210
1247899642,43cyclictest0-21swapper/110:50:091
1247799648,43cyclictest24329-21grep09:10:320
1247799638,45cyclictest0-21swapper/009:16:140
1247799637,46cyclictest20522-21unixbench_singl12:20:360
1247799635,47cyclictest23267-21diskmemload11:40:310
1247899625,10cyclictest4718-21ssh10:05:201
1247899622,47cyclictest5121-21ssh11:23:591
1247899622,42cyclictest0-21swapper/109:45:251
1247899621,5cyclictest15783-21/usr/sbin/munin11:35:371
1247899621,51cyclictest0-21swapper/112:26:201
1247899621,3cyclictest0-21swapper/108:25:291
1247899621,3cyclictest0-21swapper/108:25:291
1247799628,41cyclictest0-21swapper/008:00:360
1247799628,38cyclictest0-21swapper/009:30:190
1247799624,44cyclictest26166-21ssh11:49:280
12479996118,9cyclictest7850irq/56-dwc_otg09:35:222
1247899611,4cyclictest13103-21ssh10:54:431
1247799611,47cyclictest25857-21/usr/sbin/munin10:30:280
1247899607,35cyclictest7950irq/56-dwc_otg_07:25:351
1247899601,45cyclictest14842-21ssh10:55:351
1247899601,40cyclictest8050irq/56-dwc_otg_08:00:221
1247899601,39cyclictest16841-21rm12:16:231
1247799604,48cyclictest0-21swapper/011:12:320
1247799604,44cyclictest0-21swapper/012:09:570
1247799602,44cyclictest14657-21ssh12:14:580
1247799602,42cyclictest0-21swapper/012:28:050
1247799602,41cyclictest9635-21ntpq10:10:320
12477996010,33cyclictest125-21systemd-journal11:55:090
12477996010,33cyclictest125-21systemd-journal11:55:090
1247899594,39cyclictest8050irq/56-dwc_otg_10:24:311
1247899593,41cyclictest8050irq/56-dwc_otg_10:02:051
1247899592,16cyclictest0-21swapper/109:40:201
1247899591,43cyclictest24831-21sh11:47:011
1247799595,40cyclictest0-21swapper/010:44:360
1247799593,41cyclictest0-21swapper/010:15:530
1247799592,18cyclictest0-21swapper/010:25:220
1247799591,37cyclictest0-21swapper/012:15:330
1247799591,36cyclictest29024-21wget09:55:250
1247899584,39cyclictest8050irq/56-dwc_otg_12:09:241
1247899584,32cyclictest8050irq/56-dwc_otg_09:34:071
1247899582,42cyclictest7850irq/56-dwc_otg11:15:271
1247899581,46cyclictest0-21swapper/109:55:421
1247799582,41cyclictest0-21swapper/008:10:380
1247899576,34cyclictest8050irq/56-dwc_otg_11:50:221
1247899576,34cyclictest8050irq/56-dwc_otg_11:50:221
1247899574,33cyclictest5094-21cron08:20:091
1247899572,41cyclictest0-21swapper/111:26:501
1247799577,36cyclictest0-21swapper/010:52:440
1247799576,38cyclictest8592-21ssh10:10:100
12477995712,42cyclictest101ktimersoftd/008:44:560
1247899561,43cyclictest0-21swapper/108:42:171
1247799566,41cyclictest0-21swapper/012:03:390
12477995635,7cyclictest25244-21aten_rbpower_vo07:45:230
1247799563,38cyclictest8031-21ssh11:26:450
1247799561,52cyclictest0-21swapper/009:05:210
1247899555,34cyclictest4938-21rm10:44:551
1247899553,42cyclictest8050irq/56-dwc_otg_11:41:031
1247899552,38cyclictest0-21swapper/110:35:471
1247899552,38cyclictest0-21swapper/110:35:471
12478995518,34cyclictest16786-21/usr/sbin/munin07:20:311
1247899551,7cyclictest32646-21ssh11:56:101
1247799557,34cyclictest19792-21cut07:30:240
1247799556,38cyclictest0-21swapper/010:24:480
1247799556,37cyclictest0-21swapper/008:19:460
12477995543,5cyclictest7850irq/56-dwc_otg12:30:300
1247799552,38cyclictest31796-21sh11:55:310
198822540,1sleep00-21swapper/009:00:210
1247899545,39cyclictest25232-21ssh09:50:271
1247899544,37cyclictest0-21swapper/112:23:531
1247899543,4cyclictest4383-21chrt12:01:141
1247899543,3cyclictest0-21swapper/110:30:231
1247899541,40cyclictest5338-21apt-get08:20:221
1247899541,3cyclictest0-21swapper/107:30:271
1247899541,39cyclictest8050irq/56-dwc_otg_09:38:361
1247899541,39cyclictest7950irq/56-dwc_otg_10:16:131
1247799544,38cyclictest0-21swapper/010:00:260
1247799544,14cyclictest0-21swapper/011:05:230
1247899534,38cyclictest8050irq/56-dwc_otg_11:07:181
1247899531,4cyclictest0-21swapper/111:14:021
1247899531,37cyclictest0-21swapper/108:59:361
1247899531,2cyclictest0-21swapper/110:25:261
12478995312,37cyclictest22201-21sed09:05:291
1247799537,33cyclictest0-21swapper/011:04:230
1247799534,40cyclictest24761-21runrttasks07:43:430
1247799534,40cyclictest24761-21runrttasks07:43:430
1247799531,44cyclictest0-21swapper/007:24:140
1247799531,43cyclictest0-21swapper/011:20:260
1247799531,39cyclictest30484-21cat08:00:100
1247999527,24cyclictest31873-21grep11:55:332
1247999522,7cyclictest0-21swapper/212:00:212
1247899523,39cyclictest0-21swapper/108:08:591
1247899521,47cyclictest13617-21sed07:10:381
1247799525,35cyclictest0-21swapper/008:08:530
1247799524,39cyclictest0-21swapper/008:33:490
1247799523,34cyclictest14413-21aten_rbpower_cu08:45:240
1247999516,10cyclictest8050irq/56-dwc_otg_10:30:092
1247899512,40cyclictest7950irq/56-dwc_otg_09:21:591
1247899511,6cyclictest0-21swapper/107:15:211
1247899511,4cyclictest22224-21ntp_states07:35:321
1247899511,46cyclictest30410-21latency_hist08:00:091
12477995133,8cyclictest0-21swapper/007:13:450
1247799512,36cyclictest0-21swapper/008:21:220
1248099505,9cyclictest27613-21ssh12:30:193
1247999503,18cyclictest7950irq/56-dwc_otg_11:35:152
1247899501,4cyclictest15932-21apt-get08:50:151
1247899501,4cyclictest15932-21apt-get08:50:151
1247899501,46cyclictest2842-21unixbench_multi08:10:381
1247799501,34cyclictest0-21swapper/007:25:240
1247999497,26cyclictest0-21swapper/209:30:162
12478994917,29cyclictest25443-21grep07:45:261
1247899491,37cyclictest7850irq/56-dwc_otg07:52:051
1247799496,28cyclictest0-21swapper/007:55:050
68932480,35sleep38050irq/56-dwc_otg_12:05:233
1248099482,9cyclictest8050irq/56-dwc_otg_10:40:203
1247999486,24cyclictest26001-21latency10:30:292
1247899482,43cyclictest0-21swapper/109:05:091
1247899481,44cyclictest8969-21basename08:30:241
1247899481,43cyclictest10786-21aten_rbpower_cu08:35:281
12480994710,13cyclictest61-21kswapd011:05:203
1247999476,17cyclictest0-21swapper/210:45:222
1248099462,8cyclictest8050irq/56-dwc_otg_08:40:233
1247999467,25cyclictest0-21swapper/209:20:212
12479994626,12cyclictest301ktimersoftd/209:05:212
12479994612,10cyclictest0-21swapper/209:50:222
1247899462,33cyclictest0-21swapper/107:41:011
1247899462,33cyclictest0-21swapper/107:41:011
12479994522,8cyclictest8050irq/56-dwc_otg_11:01:362
12479994521,4cyclictest18036-21aten_rbpower_vo07:25:252
12479994514,23cyclictest7950irq/56-dwc_otg_09:43:552
1248099441,8cyclictest0-21swapper/310:25:243
1247999447,25cyclictest0-21swapper/211:40:222
120302442,36sleep00-21swapper/007:07:580
1247999439,11cyclictest14975-21ssh10:55:362
1247999435,12cyclictest29542-21ssh09:16:142
1247999431,9cyclictest0-21swapper/210:15:202
1247999428,20cyclictest333-21irqstats08:05:282
1247999426,11cyclictest10382-21ssh12:09:402
12479994214,15cyclictest8050irq/56-dwc_otg_10:53:082
305682410,1sleep20-21swapper/209:56:152
1248099415,29cyclictest0-21swapper/307:30:223
12480994119,8cyclictest8509-21fschecks_time09:30:243
1247999419,10cyclictest15148-21cut12:15:222
1247999418,18cyclictest0-21swapper/210:10:162
1247999416,24cyclictest0-21swapper/210:00:232
1247999414,23cyclictest2998-21sshd08:11:452
1247999413,7cyclictest7950irq/56-dwc_otg_09:10:222
12479994127,7cyclictest8050irq/56-dwc_otg_11:29:392
12479994121,8cyclictest8050irq/56-dwc_otg_12:12:102
12479994118,6cyclictest7850irq/56-dwc_otg11:46:352
12479994114,11cyclictest301ktimersoftd/210:21:002
12479994112,9cyclictest0-21swapper/210:05:202
12479994110,16cyclictest3233-21ssh10:41:492
1248099407,22cyclictest14496-21ssh11:35:223
1247999408,18cyclictest5132-21cat08:20:102
1247999407,19cyclictest0-21swapper/208:50:152
1247999407,19cyclictest0-21swapper/208:50:152
1247999404,17cyclictest31683-21ssh12:35:122
1247999404,17cyclictest31683-21ssh12:35:122
1247999403,25cyclictest12274-21taskset08:39:502
12479994010,11cyclictest31212-21ssh12:34:462
12480993916,5cyclictest28896-21/usr/sbin/munin09:55:363
1247999399,10cyclictest0-21swapper/208:45:262
1247999397,10cyclictest0-21swapper/207:14:072
1247999395,18cyclictest125-21systemd-journal08:59:342
1247999394,9cyclictest0-21swapper/207:57:302
1247999392,3cyclictest0-21swapper/207:45:202
12479993919,9cyclictest7950irq/56-dwc_otg_12:29:322
12479993914,4cyclictest301ktimersoftd/207:30:242
12479993914,3cyclictest8050irq/56-dwc_otg_11:12:552
121412390,5sleep31248099cyclictest10:14:153
1248099386,6cyclictest8240-21unixbench_multi08:25:373
1248099386,6cyclictest8240-21unixbench_multi08:25:373
12480993819,4cyclictest727-21runrttasks09:19:333
1248099381,26cyclictest16807-21cut09:40:273
1247999382,26cyclictest3545-21taskset11:21:062
12479993821,8cyclictest7850irq/56-dwc_otg09:47:482
12479993815,7cyclictest7950irq/56-dwc_otg_07:35:342
1247999381,10cyclictest30679-21ssh11:55:092
1247999381,10cyclictest30679-21ssh11:55:092
1248099377,24cyclictest0-21swapper/312:00:233
1247999375,10cyclictest10201-21taskset08:31:242
1247999373,16cyclictest25487-21ssh11:09:402
1247999371,22cyclictest0-21swapper/210:35:232
1247999371,22cyclictest0-21swapper/210:35:232
1248099368,17cyclictest0-21swapper/312:20:293
1248099366,12cyclictest8050irq/56-dwc_otg_10:20:093
1248099365,25cyclictest0-21swapper/307:50:223
12480993617,13cyclictest37-21rcuc/312:35:213
12480993617,13cyclictest37-21rcuc/312:35:213
1248099361,2cyclictest0-21swapper/308:50:203
1248099361,2cyclictest0-21swapper/308:50:203
1248099361,23cyclictest15961-21ls08:50:093
12480993611,13cyclictest28648-21ssh09:55:103
1247999364,19cyclictest31950-21taskset08:03:232
1247999361,15cyclictest0-21swapper/208:40:212
1248099355,24cyclictest1978-21ssh11:20:223
1248099355,14cyclictest8050irq/56-dwc_otg_10:05:233
1248099351,3cyclictest0-21swapper/308:35:323
1247999355,7cyclictest7950irq/56-dwc_otg_07:50:212
1247999353,13cyclictest0-21swapper/207:41:222
1247999353,13cyclictest0-21swapper/207:41:222
12479993510,3cyclictest8050irq/56-dwc_otg_08:20:102
1248099349,14cyclictest0-21swapper/309:35:383
1248099345,14cyclictest8050irq/56-dwc_otg_11:40:263
1248099342,15cyclictest8050irq/56-dwc_otg_11:30:253
1248099342,15cyclictest8050irq/56-dwc_otg_11:30:253
12480993410,4cyclictest0-21swapper/312:15:263
12480993410,12cyclictest0-21swapper/310:45:133
1247999345,11cyclictest8050irq/56-dwc_otg_07:24:382
1247999343,16cyclictest21190-21chrt09:02:002
1247999342,8cyclictest0-21swapper/207:15:212
12479993412,5cyclictest0-21swapper/211:30:272
12479993412,5cyclictest0-21swapper/211:30:272
12479993412,4cyclictest7356-21/usr/sbin/munin08:25:262
12479993412,4cyclictest7356-21/usr/sbin/munin08:25:262
1248099333,6cyclictest7950irq/56-dwc_otg_11:15:373
1248099331,8cyclictest24459-21ssh09:50:093
1248099331,27cyclictest21958-21cut09:05:263
1248099331,18cyclictest18206-21cat10:20:363
1248099331,15cyclictest8050irq/56-dwc_otg_08:55:283
1248099328,4cyclictest0-21swapper/310:02:403
1248099327,22cyclictest23639-21ssh09:10:233
1248099326,12cyclictest8050irq/56-dwc_otg_10:55:353
1248099323,26cyclictest26891-21ssh11:50:213
1248099323,26cyclictest26891-21ssh11:50:213
1248099321,22cyclictest2394-21latency_hist12:00:093
1248099321,19cyclictest26628-21ssh11:50:103
1248099321,15cyclictest7950irq/56-dwc_otg_11:25:383
12480993210,4cyclictest389-21/usr/sbin/munin08:05:283
1248099316,12cyclictest7850irq/56-dwc_otg10:35:333
1248099316,12cyclictest7850irq/56-dwc_otg10:35:333
1248099315,6cyclictest8050irq/56-dwc_otg_07:15:383
12480993120,7cyclictest3301-21latency_hist08:15:093
1248099311,20cyclictest0-21swapper/308:00:223
12480993111,4cyclictest0-21swapper/310:50:233
1248099301,7cyclictest23628-21wget12:25:253
12480993017,4cyclictest18731-21ssh11:00:343
1248099301,3cyclictest8050irq/56-dwc_otg_07:25:223
12480993012,5cyclictest32433-21sort09:20:253
1248099301,24cyclictest24270-21timerandwakeup07:40:363
1248099301,24cyclictest24270-21timerandwakeup07:40:363
1248099301,22cyclictest0-21swapper/307:10:113
1248099301,15cyclictest8050irq/56-dwc_otg_11:14:183
1248099301,15cyclictest8050irq/56-dwc_otg_07:40:103
104922303,15sleep30-21swapper/307:05:103
1248099297,11cyclictest7950irq/56-dwc_otg_10:30:233
1248099292,4cyclictest7850irq/56-dwc_otg08:20:103
1248099291,12cyclictest8050irq/56-dwc_otg_07:24:063
12480992824,2cyclictest0-21swapper/307:55:293
12480992713,8cyclictest103012chrt08:32:423
1248099271,13cyclictest8050irq/56-dwc_otg_07:45:283
120572273,14sleep20-21swapper/207:08:212
1248099266,5cyclictest1992350wget09:00:243
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional