You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 16:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Aug 07, 2026 12:43:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8463991543,127cyclictest0-21swapper/011:25:020
84639915113,132cyclictest27600-21timerwakeupswit12:35:280
8463991443,100cyclictest0-21swapper/011:50:330
8463991405,111cyclictest0-21swapper/012:10:220
84639913534,73cyclictest17141-21/usr/sbin/munin09:45:350
84639913143,71cyclictest5226-21ssh10:50:210
8463991252,90cyclictest7950irq/56-dwc_otg_12:20:310
84639912424,80cyclictest0-21swapper/011:25:030
84639912318,98cyclictest937250wget11:35:160
84639912312,70cyclictest0-21swapper/009:55:220
84639912078,8cyclictest0-21swapper/010:10:270
84659911910,8cyclictest7950irq/56-dwc_otg_09:45:252
84639911862,24cyclictest7992-21fschecks_time09:35:220
84639911862,24cyclictest7992-21fschecks_time09:35:220
8463991163,10cyclictest13665-21grep11:40:200
84639911531,50cyclictest7750irq/56-dwc_otg11:15:270
84639911524,73cyclictest0-21swapper/009:40:280
8463991134,84cyclictest25311-21ssh10:35:220
8463991103,69cyclictest4399-21latency_hist10:50:020
8463991103,69cyclictest4399-21latency_hist10:50:020
84639911017,66cyclictest32318-21aten_rbpower_cu10:05:170
84659910815,24cyclictest7950irq/56-dwc_otg_12:20:062
8463991074,30cyclictest7950irq/56-dwc_otg_09:50:200
8465991063,8cyclictest0-21swapper/211:25:022
8464991062,21cyclictest7850irq/56-dwc_otg_12:10:221
8464991062,12cyclictest872-21latency_hist11:25:031
8463991043,10cyclictest21352-21ntp_states10:30:260
8463991033,56cyclictest0-21swapper/012:05:210
8463991032,12cyclictest10683-21ntp_states12:15:270
8464991023,13cyclictest17134-21ssh09:45:351
8464991013,20cyclictest7950irq/56-dwc_otg_11:50:301
84659910010,8cyclictest7850irq/56-dwc_otg_12:35:282
699921000,1sleep30-21swapper/309:34:333
846499999,9cyclictest7850irq/56-dwc_otg_10:10:291
846499997,18cyclictest7850irq/56-dwc_otg_12:05:221
846499974,9cyclictest0-21swapper/110:05:161
846599965,16cyclictest7950irq/56-dwc_otg_10:10:272
846399953,65cyclictest0-21swapper/010:20:110
846499945,14cyclictest9483-21ssh11:35:181
846599937,10cyclictest5285-21missed_timers10:50:222
846599934,10cyclictest0-21swapper/210:05:152
846599926,8cyclictest7750irq/56-dwc_otg09:35:272
846599926,8cyclictest7750irq/56-dwc_otg09:35:272
846599923,12cyclictest1881-21diskstats12:05:182
846499923,11cyclictest0-21swapper/110:50:231
846499918,9cyclictest0-21swapper/110:35:221
846499913,8cyclictest0-21swapper/112:35:281
846399913,81cyclictest30658-21ssh12:00:230
846499907,9cyclictest7750irq/56-dwc_otg09:30:151
846499904,14cyclictest7750irq/56-dwc_otg12:00:251
846599893,59cyclictest7950irq/56-dwc_otg_09:40:282
8465998812,9cyclictest7850irq/56-dwc_otg_11:50:292
846599875,22cyclictest25443-21ntpq10:35:252
846599875,11cyclictest0-21swapper/210:50:032
846599875,11cyclictest0-21swapper/210:50:032
8465998710,28cyclictest7850irq/56-dwc_otg_11:35:272
846399871,67cyclictest5153-21/usr/sbin/munin11:30:170
846499854,26cyclictest0-21swapper/109:50:211
37502850,2sleep10-21swapper/111:28:041
192532850,2sleep20-21swapper/211:07:162
8464998410,16cyclictest7950irq/56-dwc_otg_10:45:171
8464998410,16cyclictest7950irq/56-dwc_otg_10:45:171
8463998316,50cyclictest8428-21fschecks_time10:15:200
846399802,68cyclictest23940-21ssh09:15:160
8463998013,42cyclictest0-21swapper/008:47:260
306322800,2sleep20-21swapper/210:02:382
8463997934,30cyclictest0-21swapper/009:30:140
846399791,33cyclictest0-21swapper/010:25:120
846599784,26cyclictest7950irq/56-dwc_otg_12:10:232
8465997726,9cyclictest301ktimersoftd/212:15:282
846399771,56cyclictest28722-21ssh09:20:270
191392770,5sleep2846599cyclictest12:25:262
8463997620,39cyclictest22018-21idleruntime-cro12:30:000
846599757,8cyclictest7950irq/56-dwc_otg_11:15:272
8464997529,24cyclictest15307-21ssh12:20:311
8463997512,51cyclictest30486-21fschecks_time08:10:180
8463997511,44cyclictest140-21systemd-journal08:10:010
8463997511,44cyclictest140-21systemd-journal08:10:010
321052740,2sleep032066-21ssh10:05:020
846599733,13cyclictest21339-21grep10:30:242
846399731,56cyclictest13365-21unixbench_multi08:50:280
846399721,53cyclictest0-21swapper/007:46:200
846699714,46cyclictest0-21swapper/310:50:233
8463997013,48cyclictest24962-21ssh12:32:510
846399701,11cyclictest0-21swapper/011:49:240
846399692,48cyclictest0-21swapper/011:05:270
846699683,48cyclictest1885-21ssh12:05:193
846399681,16cyclictest0-21swapper/010:40:250
846499672,9cyclictest7950irq/56-dwc_otg_10:30:241
846399672,8cyclictest10333-21latency_hist08:45:010
846399671,54cyclictest21870-21diskmemload09:12:080
3962670,3sleep3402-21timerandwakeup09:25:273
28432670,2sleep20-21swapper/211:26:242
8466996616,26cyclictest7950irq/56-dwc_otg_11:20:313
8464996615,12cyclictest23-21ksoftirqd/111:15:231
8463996613,45cyclictest27384-21ssh11:55:580
846399661,14cyclictest0-21swapper/011:13:110
846399661,14cyclictest0-21swapper/011:13:110
846699653,44cyclictest867-21ssh10:45:203
846699653,44cyclictest867-21ssh10:45:203
846699653,27cyclictest0-21swapper/312:20:313
8466996511,31cyclictest7950irq/56-dwc_otg_11:15:283
846399657,40cyclictest0-21swapper/008:25:150
846399652,31cyclictest0-21swapper/007:20:120
846399651,16cyclictest22951-21/usr/sbin/munin07:50:130
846399651,14cyclictest32469-21missed_timers09:25:210
8463996510,47cyclictest0-21swapper/010:57:020
8466996415,27cyclictest7950irq/56-dwc_otg_12:15:303
846599642,47cyclictest7950irq/56-dwc_otg_09:55:222
846499643,15cyclictest0-21swapper/109:35:231
846499643,15cyclictest0-21swapper/109:35:231
846399642,48cyclictest0-21swapper/007:33:530
8463996417,38cyclictest7950irq/56-dwc_otg_07:11:200
145562640,2sleep30-21swapper/311:01:133
8466996310,12cyclictest4729-21ntp_states10:10:253
8463996313,5cyclictest0-21swapper/007:38:340
8463996313,4cyclictest0-21swapper/008:15:130
148412630,2sleep10-21swapper/109:43:531
846699623,33cyclictest39-21ksoftirqd/310:30:243
8466996216,22cyclictest7850irq/56-dwc_otg_11:40:193
846599626,12cyclictest20268-21ssh09:50:202
846399622,15cyclictest0-21swapper/011:01:570
846399621,49cyclictest0-21swapper/007:56:520
846399621,48cyclictest0-21swapper/008:55:530
846399612,31cyclictest0-21swapper/009:00:120
8463996117,36cyclictest0-21swapper/008:20:380
238552610,5sleep1846499cyclictest11:13:071
238552610,5sleep1846499cyclictest11:13:071
846699603,29cyclictest20291-21tune2fs09:50:203
8466996012,20cyclictest7750irq/56-dwc_otg09:35:223
8466996012,20cyclictest7750irq/56-dwc_otg09:35:223
8466996010,26cyclictest37-21rcuc/309:45:243
846399602,10cyclictest8454-21latency_hist08:40:010
8463996012,29cyclictest6187-21unixbench_singl08:30:310
8463996012,29cyclictest6187-21unixbench_singl08:30:310
846699594,32cyclictest7950irq/56-dwc_otg_09:55:243
8466995943,10cyclictest39-21ksoftirqd/311:35:173
846399591,16cyclictest0-21swapper/007:26:090
846699585,36cyclictest29900-21/usr/sbin/munin12:00:233
846699583,39cyclictest13012-21vmstat09:40:313
282752580,2sleep10-21swapper/108:05:021
8582570,2sleep30-21swapper/308:17:243
846699576,27cyclictest7850irq/56-dwc_otg_10:35:253
846599573,7cyclictest0-21swapper/212:00:222
846399579,40cyclictest0-21swapper/007:44:030
8466995620,9cyclictest39-21ksoftirqd/311:50:303
8466995611,10cyclictest37-21rcuc/312:35:283
846699553,18cyclictest0-21swapper/312:10:213
846699552,27cyclictest7950irq/56-dwc_otg_10:05:143
846599556,11cyclictest12197-21apt-get10:20:122
846599553,15cyclictest3717-21ssh09:30:142
8465995521,28cyclictest13652-21fschecks_count11:40:202
846499556,16cyclictest0-21swapper/111:45:071
846499553,12cyclictest13603-21diskstats11:40:181
8464995415,19cyclictest7750irq/56-dwc_otg09:55:201
846399545,3cyclictest0-21swapper/007:20:010
846499537,28cyclictest0-21swapper/107:45:141
846399538,5cyclictest28121-21idleruntime-cro08:05:010
846499524,17cyclictest0-21swapper/110:20:141
846499515,17cyclictest7950irq/56-dwc_otg_12:15:291
8464995127,9cyclictest7850irq/56-dwc_otg_09:10:211
846399511,15cyclictest18957-21chrt09:06:390
8466995015,10cyclictest39-21ksoftirqd/310:20:123
846499503,4cyclictest0-21swapper/109:05:111
8466994923,5cyclictest10254-21latency_hist08:45:013
846699486,11cyclictest7750irq/56-dwc_otg11:45:153
846699483,6cyclictest7850irq/56-dwc_otg_09:20:123
846699483,4cyclictest0-21swapper/308:25:133
846499486,33cyclictest7950irq/56-dwc_otg_10:25:121
846599477,28cyclictest0-21swapper/210:15:162
81592473,12sleep00-21swapper/007:09:350
846599457,24cyclictest17492-21ssh11:45:072
846499456,25cyclictest12868-21ssh11:00:141
846699443,5cyclictest7750irq/56-dwc_otg08:10:123
846499441,11cyclictest0-21swapper/108:20:131
846699437,20cyclictest886-21ssh11:25:033
846599437,23cyclictest0-21swapper/207:45:142
846499434,10cyclictest31947-21apt10:05:011
846499426,23cyclictest0-21swapper/108:25:071
93052410,1sleep20-21swapper/207:10:252
846599417,24cyclictest0-21swapper/209:07:372
846599413,10cyclictest7950irq/56-dwc_otg_08:05:182
846599413,10cyclictest7950irq/56-dwc_otg_08:05:182
8465994119,8cyclictest2657450wget08:00:142
846499416,4cyclictest20673-1kworker/u9:112:30:191
8464994114,17cyclictest2264-1kworker/u9:110:56:171
72462417,20sleep10-21swapper/107:05:231
186192410,1sleep10-21swapper/111:06:061
8466994018,5cyclictest12658-21latency_hist11:00:013
846499407,23cyclictest0-21swapper/110:15:161
846499407,22cyclictest0-21swapper/108:50:141
8466993917,5cyclictest24849-21unixbench-2d09:15:283
846699391,24cyclictest0-21swapper/307:40:133
8464993917,6cyclictest5753-21/usr/sbin/munin11:30:221
8464993916,8cyclictest1583950wget07:30:161
846699388,20cyclictest0-21swapper/312:25:153
846699387,4cyclictest7750irq/56-dwc_otg08:05:193
846699387,4cyclictest7750irq/56-dwc_otg08:05:193
846699385,3cyclictest0-21swapper/308:55:113
846699381,23cyclictest1932-21missed_timers08:20:213
846499382,15cyclictest7950irq/56-dwc_otg_08:45:301
8464993817,7cyclictest15963-21aten_rbpower_po09:00:151
76682387,18sleep37788-21grep07:05:293
8466993713,6cyclictest6270-21cat11:30:293
846599377,5cyclictest7850irq/56-dwc_otg_09:15:012
846599376,13cyclictest7950irq/56-dwc_otg_07:40:282
846499376,6cyclictest0-21swapper/109:25:281
846499374,13cyclictest7950irq/56-dwc_otg_07:40:271
846499371,6cyclictest1191-21snmpd09:15:221
846499371,2cyclictest0-21swapper/110:40:141
846499371,27cyclictest0-21swapper/108:30:261
846499371,27cyclictest0-21swapper/108:30:261
846499371,26cyclictest0-21swapper/111:59:271
846699368,24cyclictest0-21swapper/307:50:133
846699367,26cyclictest0-21swapper/309:00:143
846699367,24cyclictest0-21swapper/309:05:123
8466993615,5cyclictest24730-21/usr/sbin/munin07:55:273
846699361,27cyclictest16736-21timerwakeupswit07:30:303
8465993624,8cyclictest301ktimersoftd/210:55:132
8465993616,4cyclictest5704-21uname11:30:212
846499369,7cyclictest7950irq/56-dwc_otg_07:10:271
846499368,25cyclictest0-21swapper/107:35:151
112752361,3sleep20-21swapper/207:15:292
846699357,17cyclictest381ktimersoftd/310:40:143
846699351,23cyclictest22547-21ssh12:30:163
846699351,19cyclictest8609-21ls08:40:013
846599354,13cyclictest22371-21sh12:30:142
8465993517,6cyclictest5859-21taskset08:30:282
8465993517,6cyclictest5859-21taskset08:30:282
846499356,14cyclictest7950irq/56-dwc_otg_09:20:221
846499351,26cyclictest0-21swapper/107:30:011
8466993427,5cyclictest17111-21aten_rbpower_vo11:05:173
846699341,22cyclictest0-21swapper/308:48:333
846599349,5cyclictest7850irq/56-dwc_otg_08:45:122
846599346,21cyclictest7950irq/56-dwc_otg_07:35:152
8465993417,10cyclictest13929-21aten_rbpower_cu07:25:152
846499347,24cyclictest0-21swapper/107:20:141
846499345,4cyclictest0-21swapper/108:40:011
846499343,17cyclictest7750irq/56-dwc_otg08:40:161
846699337,5cyclictest0-21swapper/309:12:063
846699333,17cyclictest7850irq/56-dwc_otg_07:15:183
846699333,14cyclictest19284-21diskmemload10:25:293
846699331,21cyclictest0-21swapper/308:30:023
846699331,21cyclictest0-21swapper/308:30:023
846599337,5cyclictest3147-21ls08:25:022
846599337,11cyclictest7950irq/56-dwc_otg_09:17:542
846599335,13cyclictest7850irq/56-dwc_otg_08:55:282
8465993315,5cyclictest28521-21sh11:58:032
8465993315,11cyclictest19284-21diskmemload09:20:332
8465993312,14cyclictest0-21swapper/211:14:012
8465993312,14cyclictest0-21swapper/211:14:012
846499334,6cyclictest7750irq/56-dwc_otg08:05:221
846499334,6cyclictest7750irq/56-dwc_otg08:05:221
846699324,7cyclictest7850irq/56-dwc_otg_07:25:233
846699322,23cyclictest0-21swapper/307:22:243
846699321,14cyclictest0-21swapper/307:35:193
846599326,15cyclictest0-21swapper/208:50:142
846599326,14cyclictest0-21swapper/208:25:072
846599325,5cyclictest7850irq/56-dwc_otg_09:00:192
846699312,6cyclictest26144-21fschecks_count11:55:203
846599317,4cyclictest7850irq/56-dwc_otg_10:25:182
846599315,9cyclictest7750irq/56-dwc_otg11:00:242
846599315,5cyclictest7850irq/56-dwc_otg_08:35:212
846599315,5cyclictest7750irq/56-dwc_otg08:40:162
846599311,5cyclictest7850irq/56-dwc_otg_09:26:062
846499315,4cyclictest7850irq/56-dwc_otg_07:50:141
846499314,5cyclictest19465-21unixbench_multi12:25:291
846499313,12cyclictest7950irq/56-dwc_otg_08:15:191
846499311,18cyclictest14238-21ls08:55:141
66152315,16sleep26301-21apt-get07:05:042
846699308,4cyclictest0-21swapper/310:15:283
846699306,5cyclictest7850irq/56-dwc_otg_11:10:183
846699306,5cyclictest7850irq/56-dwc_otg_11:10:183
846699302,5cyclictest7750irq/56-dwc_otg07:15:013
846699301,6cyclictest2111950/usr/sbin/munin07:45:183
846599308,5cyclictest31955-21/usr/sbin/munin08:15:192
846499302,5cyclictest0-21swapper/107:15:191
846499302,18cyclictest0-21swapper/107:56:271
846499301,6cyclictest7950irq/56-dwc_otg_08:15:011
846699296,14cyclictest0-21swapper/310:00:143
846599296,14cyclictest0-21swapper/207:20:142
846599292,4cyclictest0-21swapper/210:40:202
846599291,7cyclictest15713-21/usr/sbin/munin07:30:182
846699288,8cyclictest39-21ksoftirqd/308:50:283
846599282,19cyclictest0-21swapper/208:10:352
846699274,4cyclictest26602-21grep08:00:143
846599271,5cyclictest1191-21snmpd07:57:402
846599271,3cyclictest0-21swapper/207:50:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional