You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-17 - 18:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Mon Feb 17, 2025 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
280232930,2sleep121-21rcuc/110:09:571
12422998529,14cyclictest8050irq/56-dwc_otg_10:40:091
294212810,2sleep30-21swapper/310:57:223
1242299817,34cyclictest0-21swapper/107:45:111
317662790,2sleep20-21swapper/209:25:262
311752710,2sleep20-21swapper/212:25:252
258542710,2sleep20-21swapper/211:36:422
12422997123,24cyclictest21051-21unixbench-2d09:05:251
1241899713,53cyclictest5595-21tr08:25:120
1241899701,57cyclictest9851-21dpkg10:29:570
1241899701,57cyclictest9851-21dpkg10:29:570
65312680,2sleep30-21swapper/309:38:273
1241899675,39cyclictest13-21rcuc/010:00:110
1241899675,39cyclictest13-21rcuc/010:00:110
1241899671,54cyclictest0-21swapper/011:00:270
1241899671,54cyclictest0-21swapper/011:00:270
42702650,3sleep00-21swapper/010:21:350
12422996528,20cyclictest22483-21missed_timers12:15:141
303542640,1sleep1221ktimersoftd/110:14:241
12422996426,18cyclictest27304-21ssh12:20:271
12422996425,21cyclictest22230-21grep07:40:221
12422996425,12cyclictest10419-21ssh11:59:581
12422996425,12cyclictest10419-21ssh11:59:581
12418996429,3cyclictest0-21swapper/011:05:090
12422996323,22cyclictest13675-21/usr/sbin/munin10:35:161
1241899635,44cyclictest0-21swapper/010:15:200
1241899633,9cyclictest0-21swapper/012:35:040
12418996323,4cyclictest16366-21runrttasks08:55:120
12422996221,24cyclictest10169-21rm10:30:001
12418996220,5cyclictest0-21swapper/009:15:130
1241899621,58cyclictest11315-21rm09:45:160
1241899621,50cyclictest0-21swapper/009:55:010
12422996127,22cyclictest10292-21ssh11:16:041
12422996126,22cyclictest20886-21fschecks_time10:45:141
12422996125,20cyclictest10831-21rm12:41:371
1241899616,14cyclictest0-21swapper/010:55:060
324762600,2sleep00-21swapper/008:10:130
12422996032,9cyclictest19754-21apt-get07:35:131
12422996027,13cyclictest8161-21ssh12:39:151
12422996025,19cyclictest14639-21rm11:21:291
12422996022,20cyclictest6259-21wc08:25:221
12422996022,20cyclictest10639-21chmod11:59:581
12418996011,29cyclictest4495-21latency_hist12:34:570
1242999592,44cyclictest8050irq/56-dwc_otg_12:04:573
1242999591,44cyclictest0-21swapper/311:17:333
12422995931,12cyclictest2714750ssh10:55:041
12422995924,18cyclictest24438-21ssh10:50:041
12422995924,14cyclictest31286-21ssh11:44:151
12422995919,24cyclictest1633-21ssh11:46:481
12418995942,4cyclictest0-21swapper/011:35:070
1241899592,47cyclictest0-21swapper/009:24:580
1242999582,44cyclictest8150irq/56-dwc_otg_12:07:353
12422995825,12cyclictest27146-21ssh11:39:071
12422995823,11cyclictest31265-21ntpq08:05:191
12422995821,13cyclictest20242-21munin-run09:59:571
12422995816,15cyclictest13838-21grep08:45:261
1241899589,36cyclictest18741-21ls09:00:170
12418995836,15cyclictest29786-21ls12:24:570
1241899583,39cyclictest23113-21kworker/0:111:40:490
1241899582,30cyclictest13988-21unixbench_singl11:20:230
1241899581,47cyclictest16644-21apt-config09:54:570
1241899581,47cyclictest0-21swapper/009:14:570
1242999571,42cyclictest0-21swapper/310:37:533
12422995726,20cyclictest1918-21ssh09:30:331
12422995724,15cyclictest10172-21ssh09:43:511
12422995724,15cyclictest10172-21ssh09:43:511
12422995720,20cyclictest6021-21latency_hist10:24:571
12422995718,12cyclictest5161-21latency_hist08:24:581
12422995714,23cyclictest8150irq/56-dwc_otg_09:25:101
1241899573,36cyclictest0-21swapper/010:40:160
12418995710,44cyclictest9-21ksoftirqd/009:29:570
83782560,2sleep20-21swapper/211:56:182
83782560,2sleep20-21swapper/211:56:182
1242999562,37cyclictest8150irq/56-dwc_otg_09:05:163
1242999561,6cyclictest8150irq/56-dwc_otg_10:45:253
12422995626,20cyclictest28764-21munin-node08:00:081
12422995623,15cyclictest20590-21aten_rbpower_cu10:00:131
12422995623,15cyclictest20590-21aten_rbpower_cu10:00:131
12422995618,21cyclictest23360-21apt-get09:15:031
12418995634,5cyclictest8150irq/56-dwc_otg_10:05:090
12418995634,5cyclictest8050irq/56-dwc_otg_11:55:170
12418995634,5cyclictest8050irq/56-dwc_otg_11:55:170
1241899562,5cyclictest31398-21grep12:25:260
1241899561,43cyclictest0-21swapper/009:35:060
169642550,3sleep3381ktimersoftd/307:25:243
1242999559,43cyclictest10138-21grep12:40:273
1242999558,36cyclictest8150irq/56-dwc_otg_10:02:053
1242999558,36cyclictest8150irq/56-dwc_otg_10:02:053
1242999551,9cyclictest8050irq/56-dwc_otg_08:00:173
1242999551,9cyclictest23052-21unixbench_multi12:15:213
1242999551,4cyclictest0-21swapper/309:43:123
1242999551,4cyclictest0-21swapper/309:43:123
1242999551,40cyclictest0-21swapper/311:12:123
12422995521,19cyclictest6375-21ssh11:10:591
12422995519,21cyclictest19004-21ssh12:10:151
12422995519,21cyclictest19004-21ssh12:10:151
1241899554,40cyclictest23344-21/usr/sbin/munin07:45:230
12418995519,33cyclictest101ktimersoftd/009:09:570
12418995511,32cyclictest20864-21unixbench_multi07:35:290
1242999542,4cyclictest0-21swapper/310:20:113
1242999542,42cyclictest0-21swapper/311:01:323
1242999542,42cyclictest0-21swapper/311:01:323
1242999542,35cyclictest0-21swapper/307:51:313
1242999541,40cyclictest15934-21rm11:23:513
12422995422,17cyclictest8571-21ssh10:27:241
12422995422,17cyclictest8571-21ssh10:27:241
12422995420,17cyclictest3037-21taskset08:16:251
12422995420,14cyclictest1623-21ssh10:19:081
12422995414,30cyclictest0-21swapper/107:29:331
12418995429,5cyclictest8050irq/56-dwc_otg_08:40:130
1241899541,44cyclictest15441-21cat12:05:170
1241899541,11cyclictest8065-21grep09:40:190
1241899541,11cyclictest8065-21grep09:40:190
1242999532,6cyclictest28192-21ssh11:40:163
1242999532,6cyclictest0-21swapper/308:20:093
1242999531,8cyclictest0-21swapper/311:35:043
12429995314,36cyclictest27416-21aten_rbpower_vo09:20:123
1242999531,38cyclictest0-21swapper/311:32:193
1242999531,13cyclictest0-21swapper/310:08:403
12422995325,7cyclictest3741-21ssh12:33:391
12422995322,11cyclictest30291-21ssh09:24:431
12422995322,11cyclictest15342-21ssh09:52:241
12422995320,12cyclictest18733-21sort12:09:581
12422995319,21cyclictest11573-21ls09:45:191
12422995318,15cyclictest23523-21ssh11:34:571
12422995317,10cyclictest0-21swapper/112:25:441
12422995316,10cyclictest7950irq/56-dwc_otg11:01:141
12422995316,10cyclictest7950irq/56-dwc_otg11:01:141
1241899538,37cyclictest3770-21taskset09:34:460
1241899532,27cyclictest944450ssh11:15:170
12418995313,23cyclictest0-21swapper/012:14:580
12418995313,23cyclictest0-21swapper/012:14:580
1242999529,40cyclictest19729-21latency_hist09:04:573
1242999521,9cyclictest8150irq/56-dwc_otg_09:19:433
1242999521,6cyclictest17281-21fschecks_count09:55:133
1242999521,4cyclictest8150irq/56-dwc_otg_09:54:573
1242999521,41cyclictest0-21swapper/309:30:053
1242999521,3cyclictest8150irq/56-dwc_otg_09:26:203
1242999521,3cyclictest0-21swapper/311:59:573
1242999521,3cyclictest0-21swapper/311:59:573
1242999521,3cyclictest0-21swapper/311:48:033
12422995213,16cyclictest20008-21rm11:28:501
12418995222,17cyclictest23140-21latency_hist07:44:570
1241899521,8cyclictest9004-21aten_rbpower_vo12:40:130
1241899521,7cyclictest16797-21ntp_kernel_pll_07:25:200
1241899521,39cyclictest0-21swapper/007:15:110
1242999517,34cyclictest0-21swapper/312:23:423
1242999514,3cyclictest0-21swapper/309:45:523
1242999512,7cyclictest18687-21proc_pri07:30:213
1242999512,7cyclictest18687-21proc_pri07:30:213
1242999512,6cyclictest8150irq/56-dwc_otg_12:31:103
1242999512,6cyclictest21819-21grep07:40:173
1242999511,6cyclictest31648-21grep08:05:253
1242999511,4cyclictest8150irq/56-dwc_otg_08:45:243
1242999511,4cyclictest14543-21/usr/sbin/munin08:50:193
1242999511,4cyclictest14543-21/usr/sbin/munin08:50:193
12429995114,34cyclictest14565-21fschecks_count07:20:143
1242999511,33cyclictest12643-21munin-run10:34:573
12422995126,11cyclictest23039-21sshd09:11:101
12422995122,12cyclictest13936-21runrttasks07:18:401
12422995117,19cyclictest1189-21taskset08:10:581
12422995117,15cyclictest6655-21rm09:38:451
12422995117,12cyclictest0-21swapper/107:34:571
12422995117,12cyclictest0-21swapper/107:34:571
12422995115,25cyclictest6185-21ssh11:54:501
1241899511,4cyclictest15218-21chrt10:37:220
12418995111,6cyclictest8050irq/56-dwc_otg_07:30:080
12418995111,6cyclictest8050irq/56-dwc_otg_07:30:080
1242999501,8cyclictest0-21swapper/312:13:513
1242999501,8cyclictest0-21swapper/312:13:513
1242999501,7cyclictest23418-21grep07:45:113
12429995015,32cyclictest948-21sh12:29:063
1242999501,4cyclictest0-21swapper/310:42:383
1242999501,40cyclictest0-21swapper/308:30:503
12423995023,4cyclictest25169-21sh10:05:192
1242399501,9cyclictest8150irq/56-dwc_otg_08:25:062
12422995023,13cyclictest15794-21mv07:24:041
12422995020,10cyclictest8560-21taskset08:31:041
12422995017,22cyclictest12276-21taskset08:42:151
12422995017,12cyclictest0-21swapper/111:08:441
1241899508,17cyclictest7950irq/56-dwc_otg08:35:140
1241899501,26cyclictest14325-21modprobe08:49:570
1242999493,4cyclictest20037-21fschecks_time07:35:183
1242999492,8cyclictest6251-21cron11:54:573
1242999492,5cyclictest0-21swapper/307:55:143
1242999491,7cyclictest6328-21aten_rbpower_cu10:25:103
1242999491,7cyclictest6328-21aten_rbpower_cu10:25:103
12429994916,30cyclictest19503-21sh11:27:533
1242999491,5cyclictest8150irq/56-dwc_otg_08:25:093
1242999491,13cyclictest0-21swapper/308:16:333
12422994927,8cyclictest26653-21chrt07:54:121
12422994915,20cyclictest28290-21chrt07:56:571
1241899497,18cyclictest0-21swapper/012:00:250
1241899492,5cyclictest24078-21apt-get10:49:580
1242999481,6cyclictest31237-21fschecks_count10:15:103
1242999481,6cyclictest2432-21aten_rbpower_vo11:05:133
1242999481,5cyclictest0-21swapper/310:50:523
12429994812,32cyclictest0-21swapper/308:14:583
12422994819,15cyclictest15806-21sshd08:51:101
12422994819,15cyclictest15806-21sshd08:51:101
1241899482,31cyclictest0-21swapper/007:20:190
1241899481,19cyclictest3339-21ssh11:49:570
12418994811,28cyclictest0-21swapper/012:15:170
97332470,8sleep2301ktimersoftd/210:29:472
97332470,8sleep2301ktimersoftd/210:29:472
1242999477,26cyclictest0-21swapper/307:15:033
1242999472,5cyclictest0-21swapper/308:40:163
1242999471,7cyclictest9198-21aten_rbpower_po08:35:153
1242999471,7cyclictest5996-21grep12:35:213
1242999471,10cyclictest0-21swapper/310:13:053
1242999461,6cyclictest16672-21tune2fs08:55:183
1242399465,6cyclictest0-21swapper/209:55:042
12422994622,11cyclictest17643-21taskset08:56:351
12422994621,15cyclictest19514-21taskset09:02:211
1241899465,30cyclictest11-21rcu_preempt10:13:100
12418994622,13cyclictest0-21swapper/010:50:100
12418994620,5cyclictest9550irq/48-DMA14:55:080
1241899461,7cyclictest12417-21ssh10:34:340
12418994611,8cyclictest16983-21wget11:25:120
1242999452,8cyclictest22858-21unixbench-2d09:10:263
1241899451,30cyclictest0-21swapper/008:30:100
120452452,13sleep00-21swapper/007:14:220
1241899447,24cyclictest0-21swapper/011:54:200
1241899444,16cyclictest0-21swapper/011:13:280
1242399431,4cyclictest8150irq/56-dwc_otg_09:20:082
1241899432,35cyclictest0-21swapper/008:53:230
1241899432,35cyclictest0-21swapper/008:53:230
1241899431,20cyclictest131-21systemd-journal07:58:000
12418994310,23cyclictest23364-21diskmemload11:34:010
1242399423,18cyclictest5331-21grep09:35:222
12423994222,4cyclictest12972-21basename10:35:072
1241899427,20cyclictest0-21swapper/008:20:540
16582410,5sleep21242399cyclictest11:46:522
1242399415,23cyclictest2336450diskmemload09:40:092
1242399415,23cyclictest2336450diskmemload09:40:092
12423994118,5cyclictest242850wget10:20:122
12418994120,4cyclictest131-21systemd-journal08:18:040
1242399407,25cyclictest0-21swapper/210:00:132
1242399407,25cyclictest0-21swapper/210:00:132
12423994028,8cyclictest5503-21sh12:35:152
1242399401,25cyclictest0-21swapper/209:15:032
12422994014,13cyclictest0-21swapper/108:36:331
1241899403,20cyclictest29990-21kworker/0:108:05:240
60392380,1sleep20-21swapper/211:10:172
1242399386,27cyclictest8150irq/56-dwc_otg_07:30:082
1242399386,27cyclictest8150irq/56-dwc_otg_07:30:082
1242399385,17cyclictest8150irq/56-dwc_otg_08:35:142
1241899384,17cyclictest11-21rcu_preempt08:00:160
12423993711,8cyclictest12610-21aten_rbpower_vo07:15:112
1242399367,18cyclictest12561-21cat11:19:582
208792350,2sleep20-21swapper/212:13:452
208792350,2sleep20-21swapper/212:13:452
1242399357,22cyclictest0-21swapper/208:10:112
12423993517,5cyclictest14061-21cron09:49:562
12423993515,10cyclictest18626-21/usr/sbin/munin09:00:162
1242399351,21cyclictest4468-21latency_hist12:34:572
12423993510,18cyclictest0-21swapper/211:26:212
1242399347,25cyclictest0-21swapper/209:05:112
1242399341,5cyclictest0-21swapper/212:22:522
1242399341,26cyclictest0-21swapper/209:52:192
1242399341,21cyclictest131-21systemd-journal12:19:122
164362330,3sleep216439-21aten_rbpower_po08:55:152
1242399337,11cyclictest16847-21ssh10:40:102
1242399335,9cyclictest8050irq/56-dwc_otg_08:45:112
1242399332,10cyclictest2253-21latency_hist10:19:582
1242399331,5cyclictest23364-21diskmemload12:09:572
12423993314,4cyclictest8882-21latency_hist08:34:582
1242399331,26cyclictest0-21swapper/210:33:212
1242399331,22cyclictest15402-21switchtime08:50:232
1242399331,22cyclictest15402-21switchtime08:50:232
1242399331,17cyclictest5847-21ls11:09:582
1242399326,17cyclictest8050irq/56-dwc_otg_10:55:112
1242399325,4cyclictest8150irq/56-dwc_otg_12:00:272
12423993221,8cyclictest0-21swapper/207:40:172
12423993220,6cyclictest897850ssh12:40:132
12423993212,5cyclictest23364-21diskmemload11:52:582
1242399321,17cyclictest0-21swapper/211:04:482
1242399321,17cyclictest0-21swapper/211:04:482
114592328,13sleep10-21swapper/107:10:271
309522310,3sleep20-21swapper/211:43:382
245762310,5sleep231-21ksoftirqd/210:50:202
1242399316,19cyclictest0-21swapper/210:11:412
1242399312,12cyclictest8050irq/56-dwc_otg_11:20:132
12423993116,5cyclictest5370-21gpgv08:24:582
1242399311,27cyclictest24079-21latency_hist10:49:582
1242399307,12cyclictest0-21swapper/211:34:382
12423993013,9cyclictest10985-21aten_rbpower_en08:40:122
12423993013,8cyclictest22225-21if_wlan009:10:182
1242399301,22cyclictest0-21swapper/207:23:052
1242399301,20cyclictest16322-21cpuspeed07:25:132
1242399292,5cyclictest8150irq/56-dwc_otg_07:45:102
1242399291,18cyclictest2337-21missed_timers08:15:222
1242399291,13cyclictest8150irq/56-dwc_otg_07:35:152
1242399284,17cyclictest0-21swapper/209:30:232
12423992817,8cyclictest29077-21fschecks_time08:00:152
104902288,12sleep20-21swapper/207:09:592
118522273,15sleep30-21swapper/307:11:593
1242399268,4cyclictest26142-21cat07:50:262
1242399261,5cyclictest27787-21grep07:55:222
12423992613,4cyclictest30577-21/usr/sbin/munin08:05:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional