You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 03:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed May 06, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1568499986,83cyclictest0-21swapper/005:41:080
3552940,2sleep10-21swapper/109:51:171
3552940,2sleep10-21swapper/109:51:171
1568499867,63cyclictest0-21swapper/005:26:090
1568499867,63cyclictest0-21swapper/005:26:090
323042850,2sleep00-21swapper/008:32:200
15684998316,43cyclictest29280-21sh09:07:500
15684998221,41cyclictest7850irq/56-dwc_otg_05:11:080
15684998216,47cyclictest22476-21cut08:21:120
196402810,5sleep31568799cyclictest06:11:083
15684998116,12cyclictest19668-21ssh08:56:180
160162790,5sleep21568699cyclictest06:01:092
160162790,5sleep21568699cyclictest06:01:092
1568499791,22cyclictest17460-21ssh07:36:120
90672780,5sleep31568799cyclictest06:46:553
192292780,7sleep01568499cyclictest08:16:220
1568499782,63cyclictest0-21swapper/004:33:080
1568499781,23cyclictest0-21swapper/009:51:230
1568499781,23cyclictest0-21swapper/009:51:230
1568499766,53cyclictest0-21swapper/006:11:140
15684997612,44cyclictest3050-21sh06:40:560
24152750,4sleep32411-21ssh08:36:103
1568499741,55cyclictest0-21swapper/009:31:170
23082730,5sleep31568799cyclictest09:15:153
15684997316,3cyclictest0-21swapper/004:41:160
211872720,3sleep229-21rcuc/206:15:542
1568499719,15cyclictest1276-21cut07:55:560
1568499719,15cyclictest1276-21cut07:55:560
1568499711,18cyclictest6579-21chrt07:22:190
1568499701,14cyclictest0-21swapper/007:33:170
15684996935,19cyclictest7950irq/56-dwc_otg_05:50:540
132282690,5sleep01568499cyclictest09:27:280
1568499682,21cyclictest0-21swapper/007:19:330
1568499671,5cyclictest2571-21perl08:36:120
15684996713,6cyclictest1822-21fschecks_time07:56:140
1568499671,21cyclictest22549-21unixbench_singl06:16:220
1568499671,12cyclictest0-21swapper/008:07:240
15684996710,9cyclictest26376-21sshd05:00:310
1568499662,12cyclictest0-21swapper/009:40:090
15684996617,8cyclictest0-21swapper/009:13:400
15684996616,5cyclictest7850irq/56-dwc_otg_07:11:170
15684996614,4cyclictest3915-21awk09:16:190
15684996612,8cyclictest0-21swapper/007:06:070
1568499659,11cyclictest0-21swapper/006:32:480
15684996517,8cyclictest0-21swapper/005:06:070
15684996515,5cyclictest11827-21ssh06:51:080
1568499647,49cyclictest0-21swapper/007:42:100
1568499641,4cyclictest0-21swapper/007:50:110
1568499641,16cyclictest0-21swapper/007:05:440
1568499641,16cyclictest0-21swapper/007:05:440
119572640,2sleep31568799cyclictest08:46:413
1568499635,21cyclictest0-21swapper/009:22:540
1568499635,12cyclictest0-21swapper/009:48:310
15684996313,12cyclictest0-21swapper/009:57:180
15684996310,15cyclictest0-21swapper/009:02:280
1568499629,6cyclictest7850irq/56-dwc_otg_05:36:040
1568499629,3cyclictest0-21swapper/006:46:110
15684996246,7cyclictest0-21swapper/006:56:090
1568499622,5cyclictest0-21swapper/004:36:100
1568499621,49cyclictest0-21swapper/008:47:320
1568499621,46cyclictest0-21swapper/006:30:550
15684996213,4cyclictest908-21runrttasks05:02:580
1568499621,18cyclictest0-21swapper/008:41:420
1568599612,10cyclictest7750irq/56-dwc_otg07:16:051
1568499619,21cyclictest0-21swapper/009:41:100
1568499604,34cyclictest16109-21aten_rbpower_vo06:01:090
1568499604,34cyclictest16109-21aten_rbpower_vo06:01:090
1568499602,20cyclictest0-21swapper/006:21:500
15684996018,3cyclictest0-21swapper/008:55:540
15684996014,2cyclictest0-21swapper/008:29:200
1568499601,14cyclictest17725-21latency_hist08:15:550
1568499601,14cyclictest0-21swapper/008:01:520
1568499601,13cyclictest0-21swapper/007:30:380
291892590,4sleep21568699cyclictest05:06:182
1568499592,18cyclictest9-21ksoftirqd/006:36:120
15684995918,10cyclictest0-21swapper/005:16:050
15684995914,12cyclictest0-21swapper/004:54:490
1568499591,17cyclictest3002-21basename05:25:540
1568499591,16cyclictest13065-21/usr/sbin/munin05:51:180
15685995819,11cyclictest7850irq/56-dwc_otg_09:21:071
1568499561,15cyclictest15673-21chrt06:00:060
1568499561,13cyclictest6492-21runrttasks05:33:170
28542550,4sleep12825-21ssh08:36:161
21322550,1sleep30-21swapper/305:21:173
15684995512,8cyclictest21215-21aten_rbpower_po04:46:080
15684995512,8cyclictest21215-21aten_rbpower_po04:46:080
100962550,1sleep20-21swapper/208:45:512
1568499532,21cyclictest0-21swapper/006:08:140
1568699507,19cyclictest7950irq/56-dwc_otg_08:51:062
1568599486,4cyclictest0-21swapper/104:31:061
1568599482,7cyclictest7750irq/56-dwc_otg09:11:061
152792473,12sleep00-21swapper/004:29:170
1568599469,9cyclictest0-21swapper/109:16:071
1568599467,24cyclictest0-21swapper/106:06:081
1568599463,6cyclictest0-21swapper/106:31:051
1568599462,11cyclictest7950irq/56-dwc_otg_06:56:071
1568799456,25cyclictest0-21swapper/306:16:073
1568599457,25cyclictest0-21swapper/104:51:001
1568599453,5cyclictest7750irq/56-dwc_otg07:51:061
1568599453,5cyclictest7750irq/56-dwc_otg07:51:061
178252440,1sleep10-21swapper/108:54:481
1568799447,23cyclictest0-21swapper/305:16:083
1568799447,22cyclictest0-21swapper/305:31:003
1568699447,24cyclictest0-21swapper/208:36:082
1568699441,22cyclictest11381-21ssh06:50:542
1568599448,6cyclictest7750irq/56-dwc_otg05:46:211
15685994421,7cyclictest9519-21latency_hist08:05:551
1568799435,24cyclictest26957-21ssh06:31:003
1568699437,26cyclictest0-21swapper/209:11:082
1568699437,24cyclictest0-21swapper/206:51:072
1568599437,23cyclictest0-21swapper/105:01:061
1568599433,8cyclictest7850irq/56-dwc_otg_06:11:071
1568599432,11cyclictest7750irq/56-dwc_otg09:01:061
140982430,5sleep10-21swapper/104:26:141
1568699427,25cyclictest0-21swapper/204:36:092
129672420,2sleep20-21swapper/208:09:522
113792421,36sleep211458-21switchtime08:46:212
1568799416,24cyclictest18803-21ssh08:56:003
1568699417,24cyclictest0-21swapper/205:41:082
1568699416,25cyclictest0-21swapper/206:16:072
1568599417,22cyclictest0-21swapper/106:26:091
1568599416,25cyclictest0-21swapper/107:11:081
1568599415,4cyclictest7750irq/56-dwc_otg06:41:061
15685994118,6cyclictest5107-21vmstat09:56:251
1568599411,22cyclictest12107-21rm09:26:191
155482410,1sleep30-21swapper/307:33:333
124702410,2sleep229-21rcuc/207:30:302
1568599407,24cyclictest0-21swapper/105:11:081
1568599406,12cyclictest0-21swapper/106:01:121
1568599406,12cyclictest0-21swapper/106:01:121
1568599402,5cyclictest17634-21tr04:36:101
1568599400,32cyclictest0-21swapper/107:36:071
291622390,2sleep20-21swapper/207:50:392
1568799397,25cyclictest0-21swapper/306:56:093
1568699395,21cyclictest18815-21ssh08:56:012
1568699391,3cyclictest7950irq/56-dwc_otg_09:36:062
1568599394,4cyclictest7850irq/56-dwc_otg_07:31:211
139542390,4sleep213958-21aten_rbpower_en08:11:092
43532380,2sleep30-21swapper/309:16:243
15687993813,12cyclictest19258-21cron09:35:543
1568699385,21cyclictest21382-21ssh07:41:072
1568699384,10cyclictest7850irq/56-dwc_otg_09:01:052
1568599389,6cyclictest21-21rcuc/104:56:141
1568599387,28cyclictest22713-21latency_hist04:50:551
1568599387,28cyclictest22713-21latency_hist04:50:551
1568599383,21cyclictest7950irq/56-dwc_otg_05:16:221
268392370,15sleep27750irq/56-dwc_otg06:30:552
1568799373,9cyclictest9768-21ssh08:06:083
1568799372,6cyclictest7950irq/56-dwc_otg_09:46:063
1568699379,6cyclictest0-21swapper/207:01:252
1568699379,6cyclictest0-21swapper/207:01:252
1568599377,23cyclictest0-21swapper/108:56:081
15685993720,5cyclictest221ktimersoftd/108:21:141
15685993719,6cyclictest908-21runrttasks05:35:551
15685993711,7cyclictest23-21ksoftirqd/106:25:541
15687993620,5cyclictest31255-21ntp_states08:31:183
15687993618,6cyclictest21625-21/usr/sbin/munin07:41:113
15687993616,5cyclictest16045-21sh08:12:543
15687993612,5cyclictest0-21swapper/308:51:213
1568699363,20cyclictest0-21swapper/207:06:122
1568699363,18cyclictest0-21swapper/207:53:092
1568699363,18cyclictest0-21swapper/207:53:092
1568599369,4cyclictest0-21swapper/108:47:461
1568599366,6cyclictest23-21ksoftirqd/105:10:541
1568599366,5cyclictest23-21ksoftirqd/109:10:541
1568599365,15cyclictest7850irq/56-dwc_otg_05:26:091
1568599365,15cyclictest7850irq/56-dwc_otg_05:26:091
1568599365,12cyclictest7950irq/56-dwc_otg_04:45:551
1568599364,4cyclictest7950irq/56-dwc_otg_06:46:131
15685993616,5cyclictest908-21runrttasks05:51:111
312602350,12sleep1221ktimersoftd/109:50:041
289042350,2sleep20-21swapper/206:32:202
1568799352,9cyclictest0-21swapper/307:56:013
15687993524,7cyclictest381ktimersoftd/307:51:073
15687993524,7cyclictest381ktimersoftd/307:51:073
1568799351,24cyclictest32345-21ssh06:36:223
1568799351,20cyclictest0-21swapper/309:05:163
15687993510,5cyclictest7950irq/56-dwc_otg_07:36:203
1568699357,13cyclictest0-21swapper/209:51:142
1568699357,13cyclictest0-21swapper/209:51:142
15686993518,8cyclictest7850irq/56-dwc_otg_07:11:212
1568699351,6cyclictest31385-21ssh06:36:092
1568699351,5cyclictest5069-21ssh08:00:332
1568599359,5cyclictest7750irq/56-dwc_otg08:11:231
1568599359,4cyclictest0-21swapper/107:01:121
1568599359,4cyclictest0-21swapper/107:01:121
1568599356,12cyclictest7950irq/56-dwc_otg_06:55:541
1568599354,6cyclictest28418-21cron07:10:541
1568599353,4cyclictest7750irq/56-dwc_otg05:56:061
15685993515,5cyclictest908-21runrttasks06:38:311
67112340,2sleep30-21swapper/307:22:343
15687993415,5cyclictest24845-21/usr/sbin/munin04:56:223
1568799341,19cyclictest0-21swapper/306:26:143
1568699345,5cyclictest30535-21aten_rbpower_en08:31:082
15686993419,3cyclictest0-21swapper/209:31:082
15686993417,5cyclictest23611-21sh08:21:372
15686993414,4cyclictest17991-21sh07:36:192
15686993413,5cyclictest326650wget06:41:092
15686993410,5cyclictest5676-21/usr/sbin/munin07:21:212
1568599347,6cyclictest7850irq/56-dwc_otg_09:40:541
1568599345,5cyclictest8963-21aten_rbpower_en07:26:081
1568599345,4cyclictest0-21swapper/107:41:221
1568599343,15cyclictest7850irq/56-dwc_otg_08:06:201
1568599342,7cyclictest21965-21latency_hist08:20:551
1568599341,7cyclictest5899-21cat07:21:231
15685993413,5cyclictest0-21swapper/109:31:081
15685993411,12cyclictest0-21swapper/107:50:271
1568799337,5cyclictest20615-21tune2fs07:01:133
1568799337,5cyclictest20615-21tune2fs07:01:133
1568799336,7cyclictest7950irq/56-dwc_otg_04:46:223
1568799336,7cyclictest7950irq/56-dwc_otg_04:46:223
1568799333,3cyclictest7750irq/56-dwc_otg09:26:143
15687993327,4cyclictest3483-21fschecks_count06:41:123
15687993327,4cyclictest18014-21ls04:36:153
1568799332,21cyclictest0-21swapper/309:22:073
15687993317,4cyclictest26056-21sh08:25:543
1568799331,6cyclictest0-21swapper/306:06:063
1568799331,25cyclictest0-21swapper/305:36:113
1568699339,4cyclictest0-21swapper/208:30:552
1568699339,11cyclictest7950irq/56-dwc_otg_08:05:542
1568699337,15cyclictest0-21swapper/205:26:092
1568699337,15cyclictest0-21swapper/205:26:092
15686993327,4cyclictest27517-21ls05:01:182
1568699332,19cyclictest0-21swapper/205:24:062
1568699331,9cyclictest0-21swapper/209:42:352
1568699331,6cyclictest303550ssh09:16:082
15686993316,5cyclictest31-21ksoftirqd/206:56:082
15685993316,6cyclictest24120-21fschecks_time09:41:131
15685993310,5cyclictest21413-21apt-get06:16:051
1568799328,5cyclictest7950irq/56-dwc_otg_08:41:223
1568799324,6cyclictest5038-21unixbench_singl09:56:243
1568799321,16cyclictest7950irq/56-dwc_otg_05:50:553
15687993211,4cyclictest24798-21cat09:41:223
15687993211,4cyclictest0-21swapper/307:06:063
1568699324,13cyclictest7850irq/56-dwc_otg_09:10:542
1568699323,13cyclictest22896-21latency_hist04:50:552
1568699323,13cyclictest22896-21latency_hist04:50:552
1568699322,7cyclictest7850irq/56-dwc_otg_07:16:222
1568699322,5cyclictest7750irq/56-dwc_otg04:41:232
15686993222,3cyclictest7950irq/56-dwc_otg_09:22:062
1568699321,5cyclictest26426-21awk05:00:542
1568699321,17cyclictest17033-21ssh07:35:562
15686993211,6cyclictest17834-21aten_rbpower_cu06:06:082
1568699321,13cyclictest7750irq/56-dwc_otg06:25:472
1568599329,5cyclictest7750irq/56-dwc_otg08:26:141
1568599323,12cyclictest0-21swapper/105:36:211
1568599321,4cyclictest1394-21apt-get07:56:011
239622312,24sleep20-21swapper/204:51:222
1568799315,15cyclictest0-21swapper/307:16:083
1568799313,15cyclictest0-21swapper/305:13:313
15687993116,6cyclictest18395-21fschecks_time08:16:123
1568799311,4cyclictest15643-21awk06:55:543
1568799311,19cyclictest9190-21cut07:26:113
1568699314,7cyclictest7850irq/56-dwc_otg_05:36:212
1568699311,21cyclictest6906-21ssh09:59:422
1568699311,13cyclictest7950irq/56-dwc_otg_05:46:112
15686993110,11cyclictest0-21swapper/208:16:232
1568599315,7cyclictest32741-21ssh08:33:081
1568599313,6cyclictest6512-21grep08:41:081
1568799305,6cyclictest7850irq/56-dwc_otg_09:06:263
1568799304,5cyclictest39-21ksoftirqd/307:11:073
1568799302,7cyclictest7750irq/56-dwc_otg05:41:233
15687993014,6cyclictest0-21swapper/309:36:063
1568799301,23cyclictest13258-21ls05:51:213
1568799301,23cyclictest0-21swapper/306:04:393
1568799301,23cyclictest0-21swapper/306:04:393
15687993010,4cyclictest0-21swapper/309:51:173
15687993010,4cyclictest0-21swapper/309:51:173
1568699302,5cyclictest31739-21munin-run09:50:552
1568699301,8cyclictest32102-21wget05:16:082
15685993017,8cyclictest1747-21fschecks_time05:21:111
1568799298,5cyclictest7950irq/56-dwc_otg_05:01:033
1568799297,6cyclictest0-21swapper/308:26:113
1568799296,10cyclictest0-21swapper/308:01:193
1568799293,15cyclictest0-21swapper/305:29:523
1568799293,15cyclictest0-21swapper/305:29:523
1568799292,17cyclictest0-21swapper/304:53:223
1568799291,3cyclictest0-21swapper/305:06:233
1568699297,4cyclictest7950irq/56-dwc_otg_09:26:092
1568699294,6cyclictest7750irq/56-dwc_otg05:11:142
1568699293,11cyclictest7850irq/56-dwc_otg_05:55:202
1568699291,6cyclictest7750irq/56-dwc_otg05:56:142
1568599297,5cyclictest23-21ksoftirqd/105:41:161
1568799283,5cyclictest7950irq/56-dwc_otg_04:31:113
1568799282,11cyclictest7850irq/56-dwc_otg_07:46:143
15687992813,8cyclictest908-21runrttasks04:44:413
15686992817,7cyclictest15990-21fschecks_count04:31:122
1568799272,12cyclictest7950irq/56-dwc_otg_06:21:183
15686992710,5cyclictest6180-21cat05:31:222
151862274,15sleep20-21swapper/204:28:062
142942275,13sleep314347-21cut04:26:173
1568799263,11cyclictest7950irq/56-dwc_otg_05:56:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional