You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 13:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Aug 13, 2026 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2667221740,5sleep3703299cyclictest23:55:253
604721550,2sleep2703099cyclictest21:33:442
7029991103,78cyclictest0-21swapper/100:20:351
70309910547,32cyclictest15491-21grep00:20:332
70289910117,47cyclictest13475-21unixbench_singl22:20:350
7028991009,53cyclictest18841-21kworker/0:221:50:230
70289910030,54cyclictest2307-21ls00:05:260
702999993,7cyclictest0-21swapper/122:45:181
7029999912,63cyclictest12918-21/usr/sbin/munin23:00:231
7028999940,35cyclictest23705-21grep00:30:340
263812990,2sleep00-21swapper/022:36:230
174342980,14sleep27850irq/56-dwc_otg_23:45:012
174342980,14sleep27850irq/56-dwc_otg_23:45:012
7030999643,28cyclictest13444-21interrupts23:00:232
702899933,63cyclictest0-21swapper/000:00:230
7028999238,32cyclictest25594-21diskstats23:15:220
702899923,64cyclictest0-21swapper/022:45:180
702899923,63cyclictest7027-21cyclictest19:10:060
7032999110,11cyclictest15569-21ssh00:20:343
7028999138,30cyclictest5952-21rm23:30:250
7028999133,23cyclictest0-21swapper/023:05:310
703299908,22cyclictest0-21swapper/322:20:333
702999903,60cyclictest0-21swapper/123:05:311
702999893,60cyclictest20093-21rm21:50:231
702899896,41cyclictest13-21rcuc/022:55:140
7028998931,28cyclictest20803-21fschecks_time22:30:220
702899884,54cyclictest0-21swapper/023:00:210
702999872,62cyclictest7750irq/56-dwc_otg00:30:341
7030998636,35cyclictest0-21swapper/223:15:202
702899868,13cyclictest7850irq/56-dwc_otg_00:20:350
703299852,8cyclictest13418-21ssh23:00:233
702999853,75cyclictest20808-21fschecks_time22:30:221
7030998452,26cyclictest18180-21/usr/sbin/munin23:05:322
702999843,59cyclictest0-21swapper/100:05:271
702899841,47cyclictest0-21swapper/019:30:160
703299834,19cyclictest20204-21memory21:50:263
702999834,63cyclictest0-21swapper/123:30:271
7030998260,16cyclictest20720-21diskstats22:30:202
702999823,58cyclictest0-21swapper/123:15:201
702899823,45cyclictest7950irq/56-dwc_otg_23:45:160
702999814,71cyclictest30573-21sh00:00:221
7030998052,22cyclictest2404-21ssh00:05:282
34072800,5sleep2703099cyclictest22:10:002
702999794,68cyclictest13433-21ssh22:20:351
703299775,20cyclictest0-21swapper/300:00:233
326542770,2sleep10-21swapper/123:23:551
326542770,2sleep10-21swapper/123:23:551
288282770,5sleep2703099cyclictest20:10:162
703299763,16cyclictest23790-21unixbench_singl00:30:343
702999763,66cyclictest0-21swapper/119:10:061
7030997545,24cyclictest5956-21ssh23:30:262
7030997545,24cyclictest20136-21cat21:50:242
702999756,53cyclictest23800-21ssh21:55:151
702999756,53cyclictest23800-21ssh21:55:151
275242750,6sleep1702999cyclictest00:35:291
254962750,1sleep30-21swapper/321:18:243
703299743,12cyclictest20727-21diskstats22:30:213
7030997454,15cyclictest533-21apt-get22:45:162
7030997310,40cyclictest30581-21ssh00:00:222
702899736,52cyclictest0-21swapper/022:50:160
153282730,4sleep2703099cyclictest22:23:532
702899725,23cyclictest0-21swapper/000:35:150
7028997225,6cyclictest0-21swapper/022:15:140
7030997142,24cyclictest23833-21taskset00:30:362
703299706,41cyclictest7950irq/56-dwc_otg_00:05:283
703299697,36cyclictest25580-21/usr/sbin/munin23:15:203
7030996926,25cyclictest9906-21latency_hist00:15:052
7030996926,25cyclictest9906-21latency_hist00:15:052
703299683,17cyclictest22909-21cat23:50:293
7029996744,18cyclictest7950irq/56-dwc_otg_22:35:161
702999667,44cyclictest7950irq/56-dwc_otg_19:30:311
7028996619,25cyclictest32259-21grep21:25:330
7029996510,5cyclictest7950irq/56-dwc_otg_19:25:171
702899652,6cyclictest0-21swapper/023:35:150
703099643,7cyclictest7750irq/56-dwc_otg19:55:152
702999649,43cyclictest0-21swapper/123:45:141
702999646,9cyclictest7950irq/56-dwc_otg_20:50:141
702999644,46cyclictest7950irq/56-dwc_otg_23:50:271
702999644,10cyclictest7750irq/56-dwc_otg21:33:431
7028996436,6cyclictest7850irq/56-dwc_otg_00:10:320
7028996436,6cyclictest7850irq/56-dwc_otg_00:10:320
194452640,2sleep00-21swapper/021:10:320
194452640,2sleep00-21swapper/021:10:320
7030996331,17cyclictest13232-21/usr/sbin/munin19:25:292
702999634,4cyclictest7750irq/56-dwc_otg22:09:051
702999634,4cyclictest31986-21ssh21:25:301
702999631,50cyclictest27560-21ssh23:55:541
702899633,8cyclictest0-21swapper/021:45:160
7028996332,13cyclictest7750irq/56-dwc_otg22:25:320
702899632,5cyclictest3883-21ssh22:10:140
7030996225,11cyclictest7392-21rm22:14:312
7030996223,22cyclictest27715-21ssh22:38:492
7030996222,23cyclictest10714-21cat00:15:252
702999626,4cyclictest7750irq/56-dwc_otg19:45:291
702999626,4cyclictest7750irq/56-dwc_otg19:45:291
702999624,3cyclictest7750irq/56-dwc_otg23:35:241
7028996240,5cyclictest7750irq/56-dwc_otg00:25:170
7032996012,25cyclictest0-21swapper/323:05:323
7030996036,9cyclictest17792-21apt-get23:45:142
7030996028,13cyclictest32312-21ssh22:44:392
7030996027,18cyclictest17485-21ssh21:46:542
7030996025,24cyclictest22251-21latency_hist21:15:042
7030996025,24cyclictest22251-21latency_hist21:15:042
702999606,42cyclictest0-21swapper/122:01:371
702899604,23cyclictest0-21swapper/022:05:150
702899601,37cyclictest20915-21tail19:45:330
702899601,37cyclictest20915-21tail19:45:330
702899601,35cyclictest0-21swapper/021:40:150
702899601,35cyclictest0-21swapper/021:40:150
702899601,33cyclictest21694-21aten_rbpower_po19:50:170
7032995919,10cyclictest7950irq/56-dwc_otg_23:20:183
7032995919,10cyclictest7950irq/56-dwc_otg_23:20:183
7030995925,21cyclictest9563-21tr22:55:252
702999594,44cyclictest7850irq/56-dwc_otg_23:15:041
702999591,10cyclictest0-21swapper/122:50:161
7029995910,39cyclictest0-21swapper/123:41:101
7029995910,39cyclictest0-21swapper/123:41:101
702899597,27cyclictest0-21swapper/021:15:180
702899591,37cyclictest22276-21tune2fs23:50:200
703299583,6cyclictest4363-21modprobe22:50:043
7030995838,15cyclictest7031-21taskset19:10:062
7030995830,13cyclictest23366-21apt-get21:55:102
7030995830,13cyclictest23366-21apt-get21:55:102
7030995829,11cyclictest8709-21ssh21:35:572
7030995823,12cyclictest21421-21taskset23:10:182
7029995817,38cyclictest19459-21grep21:10:321
7029995817,38cyclictest19459-21grep21:10:321
702999581,3cyclictest7850irq/56-dwc_otg_21:20:191
702999581,11cyclictest0-21swapper/100:19:521
702899581,44cyclictest25533-21cut20:00:220
7030995726,21cyclictest32109-21rm21:25:322
7030995725,15cyclictest19824-21aten_rbpower_cu19:45:192
7030995725,15cyclictest19824-21aten_rbpower_cu19:45:192
702999579,5cyclictest10234-21taskset19:17:391
702899572,30cyclictest0-21swapper/023:20:290
702899572,30cyclictest0-21swapper/023:20:290
702899571,30cyclictest29855-21idleruntime-cro00:00:040
703299564,31cyclictest0-21swapper/323:30:283
7030995628,13cyclictest11577-21ssh22:19:352
7030995627,15cyclictest24599-21ssh23:53:012
7030995626,19cyclictest26003-21ssh23:55:122
7030995625,12cyclictest14784-21ssh21:44:312
7030995625,12cyclictest14784-21ssh21:44:312
7030995622,12cyclictest19066-21ssh22:28:092
7030995621,11cyclictest29453-21ssh21:22:512
702999569,31cyclictest0-21swapper/121:16:471
702999567,6cyclictest0-21swapper/122:25:321
702999563,9cyclictest7950irq/56-dwc_otg_22:55:301
702999561,5cyclictest15416-21apt-get21:45:141
7029995611,5cyclictest19603-21unixbench_multi00:25:321
702899561,5cyclictest0-21swapper/019:40:160
702899561,35cyclictest0-21swapper/020:15:150
702899561,31cyclictest2048-21fschecks_count20:25:210
7030995530,9cyclictest14506-21diskstats19:30:202
7030995526,12cyclictest32033-21ssh23:22:472
7030995526,12cyclictest32033-21ssh23:22:472
7030995526,12cyclictest19895-21ssh00:25:542
7030995524,20cyclictest7850irq/56-dwc_otg_20:50:312
7030995521,16cyclictest2737-21sh23:25:582
7030995521,11cyclictest29018-21ssh22:00:332
702999556,4cyclictest0-21swapper/122:12:421
702999552,10cyclictest7950irq/56-dwc_otg_21:44:191
702999552,10cyclictest7950irq/56-dwc_otg_21:44:191
702999551,5cyclictest7950irq/56-dwc_otg_20:15:301
702899556,14cyclictest0-21swapper/023:40:090
702899556,14cyclictest0-21swapper/023:40:090
7028995527,5cyclictest25085-21unixbench_singl21:55:330
7028995527,5cyclictest25085-21unixbench_singl21:55:330
702899552,39cyclictest0-21swapper/022:40:230
702899551,6cyclictest9-21ksoftirqd/019:35:310
7030995426,13cyclictest10852-21rm23:35:432
7030995418,25cyclictest7850irq/56-dwc_otg_22:50:172
702999541,45cyclictest0-21swapper/120:04:511
702999541,44cyclictest0-21swapper/122:17:231
702899542,25cyclictest0-21swapper/021:05:040
7030995322,8cyclictest30837-21df_inode20:15:202
702999533,8cyclictest7950irq/56-dwc_otg_00:12:261
702999533,8cyclictest7950irq/56-dwc_otg_00:12:261
702999532,4cyclictest0-21swapper/119:35:441
702999532,10cyclictest7850irq/56-dwc_otg_20:10:301
702999531,5cyclictest140-21systemd-journal21:06:231
702999531,14cyclictest0-21swapper/119:20:221
7028995333,7cyclictest7750irq/56-dwc_otg19:20:200
7028995329,5cyclictest13-21rcuc/022:00:280
7028995324,5cyclictest22433-21sed23:10:310
702899532,31cyclictest11914-21grep20:50:310
702899532,30cyclictest13577-21unixbench_multi19:25:320
7028995317,17cyclictest13-21rcuc/021:36:100
66042532,45sleep00-21swapper/019:08:070
7030995221,19cyclictest3055-21taskset20:26:022
702999521,5cyclictest1180-21runrttasks23:28:251
702999521,3cyclictest0-21swapper/121:37:581
702899529,24cyclictest0-21swapper/021:30:240
7028995237,5cyclictest101ktimersoftd/023:25:290
7028995235,5cyclictest9-21ksoftirqd/021:25:030
702899522,29cyclictest7352-21aten_rbpower_vo20:40:170
7030995123,12cyclictest10195-21chrt19:17:152
7030995123,11cyclictest26262-21ssh21:19:432
7030995119,13cyclictest12744-21/usr/sbin/munin20:55:332
702999511,8cyclictest6491-21grep20:35:321
702999511,4cyclictest17973-21apt-get19:40:101
7028995130,5cyclictest7850irq/56-dwc_otg_00:15:270
7028995121,17cyclictest9-21ksoftirqd/020:55:200
703299503,8cyclictest0-21swapper/320:30:183
703299503,8cyclictest0-21swapper/320:30:183
7030995023,12cyclictest140-21systemd-journal20:45:032
7030995023,11cyclictest9587-21sed20:45:232
7030995020,12cyclictest29534-21ssh00:38:282
702999502,9cyclictest14602-21/usr/sbin/munin21:00:161
702999501,12cyclictest0-21swapper/120:34:281
702999501,12cyclictest0-21swapper/120:34:281
702999501,11cyclictest0-21swapper/122:43:511
702999492,8cyclictest24551-21grep19:55:311
702999492,3cyclictest7850irq/56-dwc_otg_20:07:301
702999491,5cyclictest7850irq/56-dwc_otg_20:58:181
702999491,3cyclictest0-21swapper/120:40:271
702999491,10cyclictest0-21swapper/120:47:521
702899491,32cyclictest0-21swapper/020:05:080
7030994822,16cyclictest17612-21sshd21:06:172
7030994821,13cyclictest10684-21runrttasks19:20:142
7030994819,12cyclictest26774-21latency_hist20:05:042
7028994810,6cyclictest1091-21grep20:20:320
7030994718,18cyclictest0-21swapper/220:36:442
7030994716,14cyclictest5186-21latency_hist20:35:052
7030994716,14cyclictest5186-21latency_hist20:35:052
702999478,3cyclictest7950irq/56-dwc_otg_20:21:101
702999473,9cyclictest7850irq/56-dwc_otg_20:26:081
702899472,25cyclictest17974-21ssh21:10:040
703299466,14cyclictest0-21swapper/319:40:103
7030994618,13cyclictest19427-21taskset19:43:332
7030994615,21cyclictest22906-21taskset19:51:572
702899469,10cyclictest9450irq/48-DMA08:56:170
702899462,28cyclictest6177-21ps20:35:290
702899461,26cyclictest0-21swapper/020:10:150
703299452,4cyclictest7950irq/56-dwc_otg_19:15:143
703299452,3cyclictest0-21swapper/319:50:143
7030994519,12cyclictest1191-21snmpd20:07:462
7030994517,18cyclictest12042sleep220:20:472
7030994516,11cyclictest9550irq/80-mmc019:35:182
702999453,4cyclictest0-21swapper/119:51:551
703299444,7cyclictest7950irq/56-dwc_otg_19:10:143
703299442,6cyclictest7850irq/56-dwc_otg_20:10:143
7028994433,5cyclictest101ktimersoftd/020:50:040
7028994318,4cyclictest0-21swapper/020:00:040
64712430,2sleep30-21swapper/321:34:303
702899421,25cyclictest0-21swapper/020:31:130
702899421,25cyclictest0-21swapper/020:31:130
703299412,26cyclictest0-21swapper/323:35:323
703299411,25cyclictest21432-21rm23:49:463
7030994127,12cyclictest15112-21ls21:00:222
7032994023,5cyclictest732450wget20:40:173
703299386,14cyclictest7950irq/56-dwc_otg_19:40:043
703299376,12cyclictest7950irq/56-dwc_otg_19:30:043
703299371,3cyclictest24591-21ssh22:35:143
703299361,26cyclictest12597-21apt-key20:55:043
703299361,24cyclictest13359-21ls20:55:263
703299361,16cyclictest13586-21sshd00:19:283
703299357,6cyclictest0-21swapper/322:50:313
703299356,18cyclictest28647-21ls20:10:043
703299351,8cyclictest39-21ksoftirqd/322:06:563
703299351,7cyclictest7950irq/56-dwc_otg_23:40:103
703299351,7cyclictest7950irq/56-dwc_otg_23:40:103
703299346,5cyclictest7850irq/56-dwc_otg_21:20:283
703299345,3cyclictest7950irq/56-dwc_otg_21:55:183
703299345,3cyclictest7950irq/56-dwc_otg_21:55:183
703299345,16cyclictest17118-21ntp_states22:25:273
703299345,15cyclictest7950irq/56-dwc_otg_22:16:023
703299342,5cyclictest7850irq/56-dwc_otg_21:25:223
703299341,25cyclictest6993-21latency_hist20:40:043
703299341,22cyclictest25447-21grep20:00:213
67552345,19sleep36782-21/usr/sbin/munin19:10:043
703299337,6cyclictest7950irq/56-dwc_otg_21:10:183
703299337,6cyclictest7950irq/56-dwc_otg_21:10:183
703299336,15cyclictest7950irq/56-dwc_otg_00:10:173
703299336,15cyclictest7950irq/56-dwc_otg_00:10:173
703299333,14cyclictest7950irq/56-dwc_otg_22:00:343
703299332,24cyclictest7950irq/56-dwc_otg_23:10:143
7032993315,6cyclictest0-21swapper/322:12:543
703299331,24cyclictest0-21swapper/321:45:183
703299326,7cyclictest7750irq/56-dwc_otg21:05:233
703299322,9cyclictest19872-21aten_rbpower_po19:45:193
703299322,9cyclictest19872-21aten_rbpower_po19:45:193
7032993212,4cyclictest32585-21latency_hist22:45:043
703299319,4cyclictest0-21swapper/321:42:143
703299319,4cyclictest0-21swapper/321:42:143
703299319,10cyclictest0-21swapper/322:55:093
703299316,15cyclictest7950irq/56-dwc_otg_00:35:263
703299312,8cyclictest10964-21idleruntime-cro21:40:043
7032993110,4cyclictest0-21swapper/300:25:403
703299305,12cyclictest7950irq/56-dwc_otg_20:45:273
703299303,7cyclictest7850irq/56-dwc_otg_20:20:153
7032993021,7cyclictest2495-21ssh23:25:333
67052304,15sleep20-21swapper/219:09:272
703299292,18cyclictest0-21swapper/319:58:333
7032992921,6cyclictest32190-21cron20:20:043
703299282,12cyclictest7950irq/56-dwc_otg_21:00:203
63812283,12sleep10-21swapper/119:05:331
703299271,5cyclictest0-21swapper/319:30:153
7032992713,4cyclictest1180-21runrttasks20:26:283
703299261,3cyclictest0-21swapper/319:20:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional