You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 06:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed May 13, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
63692900,18sleep37950irq/56-dwc_otg_07:21:243
70802810,2sleep10-21swapper/108:41:181
138382810,1sleep20-21swapper/208:10:472
17543997913,8cyclictest652850kworker/u8:109:26:090
17544997739,20cyclictest25281-21grep07:05:591
17544997739,20cyclictest25281-21grep07:05:591
240252750,2sleep30-21swapper/307:03:533
240252750,2sleep30-21swapper/307:03:533
38342740,3sleep229-21rcuc/209:16:182
17544997440,8cyclictest12400-21apt-get05:46:041
17544997331,21cyclictest29479-21ssh07:50:131
17544997236,19cyclictest18243-21/usr/sbin/munin07:36:161
17544997234,19cyclictest3616-21tune2fs05:21:151
1754399721,27cyclictest0-21swapper/008:01:100
17544997136,17cyclictest25852-21ntp_states06:21:221
209712700,2sleep20-21swapper/207:39:432
17544996936,14cyclictest23347-21ntp_states08:21:251
17544996935,21cyclictest9633-21ssh09:23:041
17544996935,16cyclictest23453-21/usr/sbin/munin04:46:181
1754399698,46cyclictest0-21swapper/007:11:100
1754399694,27cyclictest0-21swapper/005:26:090
1754399694,27cyclictest0-21swapper/005:26:090
17544996831,12cyclictest23125-21latency_hist06:15:581
17543996833,7cyclictest0-21swapper/005:11:100
24962670,2sleep00-21swapper/006:37:330
17544996734,12cyclictest6768-21awk08:01:211
17544996733,23cyclictest23669-21uniq06:16:141
17544996733,23cyclictest23669-21uniq06:16:141
17544996733,20cyclictest7750irq/56-dwc_otg07:56:131
17544996732,18cyclictest5415-21ssh09:56:481
17544996732,13cyclictest8980-21ssh06:46:101
17544996729,12cyclictest4584-21cron08:37:581
17544996636,17cyclictest25812-21timerwakeupswit04:51:251
17544996636,16cyclictest11990-21ssh08:07:231
17544996634,18cyclictest32692-21ssh08:32:351
17544996632,14cyclictest11465-21ssh08:46:221
17544996630,23cyclictest25528-21ssh09:03:211
17544996627,29cyclictest28669-21ssh09:06:331
17544996626,21cyclictest7750irq/56-dwc_otg09:26:201
158432660,2sleep10-21swapper/106:54:041
17544996540,18cyclictest2303-21ssh09:53:401
17544996536,18cyclictest11918-21ssh07:28:191
17544996536,16cyclictest20746-21ssh07:00:271
17544996535,11cyclictest30503-21ssh09:48:281
1754399659,13cyclictest0-21swapper/006:16:090
1754399659,13cyclictest0-21swapper/006:16:090
1754399657,24cyclictest0-21swapper/005:31:130
17543996530,6cyclictest0-21swapper/006:11:090
17544996434,17cyclictest7251-21rm07:22:181
17544996434,16cyclictest792-21ssh09:12:121
17544996433,15cyclictest5178-21ssh09:17:281
17544996431,18cyclictest23115-21sh09:40:001
1754399642,33cyclictest0-21swapper/007:46:260
17543996413,9cyclictest7850irq/56-dwc_otg_08:46:100
17544996337,19cyclictest28310-21ssh08:27:021
17544996336,18cyclictest18444-21ssh09:34:031
17544996335,18cyclictest4602-21sed06:40:591
17544996328,19cyclictest22858-21basename09:00:581
17544996328,10cyclictest7850irq/56-dwc_otg_05:51:181
17544996325,13cyclictest29876-21taskset07:11:141
17544996323,12cyclictest7850irq/56-dwc_otg_05:15:581
17544996236,17cyclictest16615-21ssh08:13:191
17544996235,19cyclictest15832-21ssh07:32:561
17544996235,10cyclictest8136-21taskset05:32:111
17544996229,21cyclictest32577-21ssh07:53:201
17544996227,22cyclictest25027-21ssh07:44:361
17544996132,17cyclictest20917-21ssh08:18:371
17544996130,18cyclictest32391-21ssh06:35:271
17544996129,12cyclictest27772-21munin-run09:45:581
17544996126,14cyclictest2843-21ssh07:16:481
17544996036,12cyclictest8412-21ssh06:45:461
17544996036,12cyclictest8412-21ssh06:45:461
17544996032,19cyclictest27258-21ssh07:07:151
17544996031,13cyclictest1727-21fschecks_count05:16:151
17544996030,17cyclictest10007-21run-parts05:37:571
17544996026,27cyclictest0-21swapper/105:29:481
17544996026,27cyclictest0-21swapper/105:29:481
17544995930,13cyclictest22768-21idleruntime-cro04:45:581
17544995929,17cyclictest16562-21ssh08:52:021
17543995929,4cyclictest9-21ksoftirqd/009:31:120
321702580,2sleep30-21swapper/309:11:193
17546995743,10cyclictest7950irq/56-dwc_otg_06:41:073
17546995743,10cyclictest7950irq/56-dwc_otg_06:41:073
17543995730,7cyclictest7850irq/56-dwc_otg_05:45:580
17546995618,15cyclictest23653-21awk04:46:213
17544995627,12cyclictest0-21swapper/105:10:421
17543995632,7cyclictest7950irq/56-dwc_otg_09:15:580
17543995614,20cyclictest29116-21/usr/sbin/munin05:01:190
17546995511,25cyclictest7950irq/56-dwc_otg_07:31:093
17543995539,6cyclictest7850irq/56-dwc_otg_07:55:580
17544995423,10cyclictest0-21swapper/106:10:431
17543995432,5cyclictest7850irq/56-dwc_otg_09:46:220
1754399542,28cyclictest0-21swapper/009:01:230
1754399541,28cyclictest0-21swapper/006:41:160
1754399541,28cyclictest0-21swapper/006:41:160
17544995327,12cyclictest0-21swapper/104:57:131
17544995327,12cyclictest0-21swapper/104:57:131
17544995324,14cyclictest0-21swapper/105:43:211
17543995326,7cyclictest19849-21aten_rbpower_vo06:06:120
1754399531,7cyclictest26713-21munin-run06:25:570
17545995221,13cyclictest17003-21grep09:31:302
17545995220,10cyclictest8975-21apt-get06:46:102
17544995226,10cyclictest0-21swapper/104:40:371
17544995222,13cyclictest19055-21runrttasks04:35:081
17543995228,12cyclictest7950irq/56-dwc_otg_08:56:140
17543995214,5cyclictest27093-21latency04:56:180
17543995214,5cyclictest27093-21latency04:56:180
167132520,2sleep00-21swapper/006:55:410
1754699516,27cyclictest0-21swapper/304:56:113
1754699516,27cyclictest0-21swapper/304:56:113
17546995126,10cyclictest1470-21apt-get05:16:113
1754599517,30cyclictest0-21swapper/207:16:122
1754599515,29cyclictest7950irq/56-dwc_otg_07:22:262
17545995111,13cyclictest26252-21tail06:21:272
17544995141,7cyclictest16675-21ls05:56:161
17544995122,12cyclictest0-21swapper/106:26:521
17543995133,5cyclictest13961-21idleruntime-cro08:10:580
1754399512,35cyclictest0-21swapper/007:26:150
1754399512,26cyclictest0-21swapper/007:21:240
17543995119,6cyclictest18244-21fschecks_time06:01:170
17543995111,6cyclictest23290-21grep07:41:270
172202513,11sleep00-21swapper/004:30:170
17546995020,12cyclictest25746-21/usr/sbin/munin06:21:203
17546995019,16cyclictest27877-21latency_hist09:45:583
17546995016,12cyclictest2649-21ssh06:37:503
17546995015,20cyclictest17763-21ssh08:15:243
17546995010,10cyclictest7850irq/56-dwc_otg_05:11:263
17543995024,17cyclictest7950irq/56-dwc_otg_07:35:580
17543995023,5cyclictest30146-21unixbench_multi06:31:290
17543995023,15cyclictest27730-21timerandwakeup08:26:240
1754399501,47cyclictest0-21swapper/007:56:210
1754399501,30cyclictest31331-21ssh08:31:200
1754699497,26cyclictest7950irq/56-dwc_otg_09:51:223
1754699496,25cyclictest0-21swapper/305:41:113
17546994918,12cyclictest14636-21perl05:51:143
17545994914,21cyclictest9838-21ssh09:23:272
17545994914,20cyclictest4377-21ssh07:58:322
1754399496,24cyclictest0-21swapper/009:21:140
17543994926,4cyclictest18785-21ssh07:36:230
17543994925,4cyclictest765-21seq09:51:260
17543994924,6cyclictest18471-21aten_rbpower_cu08:16:120
1754399491,6cyclictest7950irq/56-dwc_otg_05:46:130
1754699488,24cyclictest7850irq/56-dwc_otg_06:57:423
1754699487,27cyclictest23191-21ssh09:01:123
17546994815,13cyclictest30159-21uname07:11:173
17546994810,19cyclictest7950irq/56-dwc_otg_05:46:113
17545994812,13cyclictest29941-21fschecks_count07:11:152
17545994812,12cyclictest32460-21df09:51:152
17544994833,11cyclictest18891-21grep06:01:261
1754399486,40cyclictest0-21swapper/005:51:100
17543994828,18cyclictest18728-21latency_hist08:55:580
17543994827,3cyclictest7950irq/56-dwc_otg_09:06:240
17543994825,3cyclictest0-21swapper/010:00:580
1754399481,6cyclictest24271-21cut09:41:160
17543994813,26cyclictest1549-21ssh07:16:130
1754399481,30cyclictest13-21rcuc/009:16:240
1754399480,31cyclictest22664-21aten_rbpower_po08:21:140
17546994716,17cyclictest16792-21wc09:31:283
17545994724,16cyclictest882-21sshd05:13:092
17545994722,12cyclictest15618-21ssh06:53:402
17545994721,18cyclictest21531-21/usr/sbin/munin06:11:202
17545994720,12cyclictest23263-21ls08:21:222
17545994714,8cyclictest20956-21munin-run04:40:582
17545994711,12cyclictest1721-21fschecks_count05:16:142
17545994710,22cyclictest25076-21ssh07:44:412
17543994725,5cyclictest7850irq/56-dwc_otg_07:01:230
17543994725,5cyclictest7850irq/56-dwc_otg_07:01:230
17543994724,12cyclictest7950irq/56-dwc_otg_08:40:590
1754399472,26cyclictest23836-21latency_hist09:40:590
17543994721,4cyclictest7950irq/56-dwc_otg_06:30:570
17543994716,8cyclictest18194-21switchtime06:56:240
17546994619,18cyclictest13301-21ssh08:09:483
17546994618,20cyclictest12906-21ssh07:30:083
17546994616,19cyclictest6920-21ssh09:20:403
17546994614,18cyclictest18177-21sh08:54:593
17546994611,19cyclictest7950irq/56-dwc_otg_09:26:193
1754599469,26cyclictest27273-21/usr/sbin/munin09:06:132
17545994619,10cyclictest6557-21apt-get08:41:102
17545994611,12cyclictest908-21runrttasks08:46:282
17544994630,13cyclictest29132-21proc_pri05:01:201
17543994625,8cyclictest7950irq/56-dwc_otg_08:41:270
1754399462,20cyclictest0-21swapper/004:41:190
1754399462,13cyclictest0-21swapper/006:00:580
290712450,3sleep329056-21ssh09:46:263
1754699456,25cyclictest0-21swapper/308:26:113
17546994517,14cyclictest3957-21tr05:21:193
17546994517,12cyclictest22761-21munin-run04:45:583
17546994517,11cyclictest6987-21ssh09:59:423
17546994517,11cyclictest16047-21ssh06:54:273
17546994510,18cyclictest7950irq/56-dwc_otg_08:04:393
17545994518,15cyclictest301ktimersoftd/204:46:242
17545994517,21cyclictest30342-21ssh06:31:422
1754399452,28cyclictest0-21swapper/008:11:190
1754399451,24cyclictest12830-21latency_hist06:50:580
1754399451,24cyclictest0-21swapper/007:06:200
1754399451,12cyclictest0-21swapper/004:36:090
262522440,6sleep21754599cyclictest07:06:212
17546994417,11cyclictest21238-21ssh08:58:013
17546994415,19cyclictest26223-21ssh08:25:473
17546994415,12cyclictest23123-21latency_hist06:15:583
17546994412,8cyclictest11996-21users08:46:283
17546994410,12cyclictest19305-21ssh07:36:423
1754599449,21cyclictest7750irq/56-dwc_otg05:51:182
1754599445,24cyclictest7950irq/56-dwc_otg_09:46:032
17545994426,14cyclictest19835-21apt-get09:36:102
17545994418,17cyclictest24005-21ssh07:03:512
17545994418,17cyclictest24005-21ssh07:03:512
17545994416,8cyclictest23562-21diskstats06:16:132
17545994416,8cyclictest23562-21diskstats06:16:132
17545994415,12cyclictest14335-21ssh09:29:072
17545994413,11cyclictest18352-21ssh08:55:182
17545994413,11cyclictest1154-21ssh08:34:162
1754599441,26cyclictest0-21swapper/205:46:032
17543994416,9cyclictest26360-21latency_hist04:55:580
1754399441,26cyclictest30529-21fschecks_time05:06:150
17546994321,9cyclictest2744650ssh07:46:293
17546994317,17cyclictest6556-21/usr/sbin/munin08:41:093
17546994315,15cyclictest27563-21fschecks_count09:06:153
17546994313,18cyclictest0-21swapper/304:51:253
17546994311,14cyclictest32361-21ssh07:52:563
1754599439,16cyclictest21664-21ssh08:58:492
1754599438,16cyclictest7850irq/56-dwc_otg_06:27:192
1754599436,14cyclictest7850irq/56-dwc_otg_05:02:202
17543994328,4cyclictest7750irq/56-dwc_otg05:16:030
17543994317,16cyclictest0-21swapper/004:34:500
166692436,24sleep116773-21unixbench-2d04:26:261
181722420,2sleep2301ktimersoftd/204:31:202
1754699429,11cyclictest32472-21ssh06:35:363
1754699425,25cyclictest7950irq/56-dwc_otg_08:34:273
17546994221,9cyclictest21572-21ssh09:37:123
17546994215,17cyclictest28174-21seq06:26:543
17546994215,14cyclictest23527-21ssh07:41:503
17546994214,11cyclictest29897-21taskset05:05:173
17546994211,20cyclictest1340-21ssh07:16:123
1754599429,19cyclictest7950irq/56-dwc_otg_07:53:052
1754599427,23cyclictest0-21swapper/209:01:122
1754599425,11cyclictest0-21swapper/208:20:402
17545994235,5cyclictest24014-21ssh09:41:122
17545994213,18cyclictest7950irq/56-dwc_otg_08:01:492
17543994219,3cyclictest0-21swapper/004:46:130
1754699413,24cyclictest7850irq/56-dwc_otg_06:46:123
17546994112,11cyclictest20735-21ssh08:18:173
17546994111,10cyclictest28528-21diskmemload09:21:143
1754599411,5cyclictest26127-21ssh07:46:112
17545994113,13cyclictest4799-21ssh08:38:202
17545994113,11cyclictest2912-21ssh06:38:162
17545994111,11cyclictest28528-21diskmemload09:59:392
1754399412,25cyclictest0-21swapper/005:40:290
1754699408,10cyclictest25425-21ssh07:06:023
1754699405,21cyclictest7950irq/56-dwc_otg_06:16:203
1754699405,21cyclictest7950irq/56-dwc_otg_06:16:203
17546994015,16cyclictest19425-21seq06:05:163
17546994014,17cyclictest5373-21ssh08:39:243
17545994016,13cyclictest3955-21ntpq05:21:192
17545994015,11cyclictest11705-21ssh07:27:562
17545994010,18cyclictest2788-21ssh09:15:502
1754699397,12cyclictest1933-21ssh07:56:103
17546993917,15cyclictest8137-21seq05:32:123
1754599399,11cyclictest31498-21chrt05:07:462
1754599395,10cyclictest0-21swapper/206:43:262
1754599395,10cyclictest0-21swapper/206:43:262
17545993910,22cyclictest17792-21ssh08:15:282
17545993910,14cyclictest17410-21ssh07:35:492
1754399396,17cyclictest0-21swapper/005:21:150
17545993814,14cyclictest19224-21ssh06:57:402
17545993810,12cyclictest25664-21sed04:51:242
17546993715,11cyclictest20966-21idleruntime-cro04:40:583
17546993714,15cyclictest0-21swapper/305:56:103
1754599377,18cyclictest0-21swapper/205:41:112
1754599376,22cyclictest0-21swapper/204:56:112
1754599376,22cyclictest0-21swapper/204:56:112
1754599373,17cyclictest301ktimersoftd/206:06:222
17546993620,12cyclictest6061-21ls05:26:243
17546993620,12cyclictest6061-21ls05:26:243
1754699361,22cyclictest0-21swapper/306:10:593
17545993611,13cyclictest28067-21ssh08:26:362
1754599351,24cyclictest21306-21wget04:41:142
17546993421,10cyclictest31562-21sshd05:08:363
1754599342,21cyclictest0-21swapper/205:56:112
1754699337,23cyclictest0-21swapper/305:36:113
1754699324,18cyclictest0-21swapper/304:31:273
1754599325,15cyclictest0-21swapper/206:04:552
17545993216,12cyclictest6180-21unixbench_multi05:26:252
17545993216,12cyclictest6180-21unixbench_multi05:26:252
17545993117,11cyclictest7357-21/usr/sbin/munin05:31:192
17545993116,12cyclictest9672-21ls05:36:252
172022298,12sleep20-21swapper/204:30:032
170112253,14sleep30-21swapper/304:27:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional