You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 11:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Dec 22, 2025 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1508999916,76cyclictest0-21swapper/020:50:200
1508999819,60cyclictest29674-21latency_hist21:20:070
1508999776,52cyclictest17880-21fschecks_time23:45:270
307062760,5sleep21509199cyclictest21:20:292
1509199767,33cyclictest0-21swapper/219:10:192
1508999764,52cyclictest0-21swapper/022:00:190
15089997510,54cyclictest16921-21idleruntime-cro20:45:050
1508999732,63cyclictest0-21swapper/021:25:280
1508999723,57cyclictest7369-21unixbench-2d20:15:360
1509099707,45cyclictest24847-21wget00:05:211
1508999697,53cyclictest0-21swapper/019:50:190
1508999696,44cyclictest21209-21cut22:25:210
1508999695,48cyclictest29959-21latency_hist22:50:060
1509099681,48cyclictest27477-21/usr/sbin/munin00:10:331
1509099671,50cyclictest22388-21/usr/sbin/munin19:30:251
145452673,14sleep00-21swapper/019:06:310
1509099661,46cyclictest32387-21/usr/sbin/munin21:25:311
1509099661,45cyclictest1491-21aten_rbpower_en00:30:211
1508999668,49cyclictest0-21swapper/019:30:270
15092996523,17cyclictest30118-21/usr/sbin/munin22:50:303
1508999656,43cyclictest0-21swapper/023:20:250
1508999656,38cyclictest0-21swapper/021:30:200
1509099641,52cyclictest21292-21ntp_states19:25:311
1509099641,40cyclictest5564-21grep20:10:381
15089996430,20cyclictest0-21swapper/020:30:330
15089996430,20cyclictest0-21swapper/020:30:330
1508999634,43cyclictest27549-21grep00:10:340
1508999628,32cyclictest0-21swapper/000:30:200
1509099611,45cyclictest2697-21apt-get20:05:161
1509099611,42cyclictest18941-21/usr/sbin/munin20:50:291
1509099601,46cyclictest7499-21/usr/sbin/munin20:15:371
1509099601,45cyclictest17854-21grep23:45:271
1509099601,42cyclictest19317-21apt-get22:20:191
1509099601,42cyclictest19317-21apt-get22:20:191
1509099601,40cyclictest29627-21apt-get19:50:191
1508999606,46cyclictest30645-21ps21:20:290
1508999605,44cyclictest0-21swapper/021:55:260
15090995921,27cyclictest17591-21if_wlan020:45:261
1509099591,45cyclictest8372-21apt-get21:50:191
1508999594,43cyclictest0-21swapper/023:06:390
15089995915,23cyclictest101ktimersoftd/020:05:120
15092995831,7cyclictest8372-21apt-get21:50:193
15090995820,35cyclictest12170-21aten_rbpower_cu23:30:201
1509099581,45cyclictest30818-21/usr/sbin/munin22:50:281
1509099581,39cyclictest11726-21apt-get20:30:181
1509099581,39cyclictest11726-21apt-get20:30:181
1508999589,18cyclictest6893-21cat23:15:200
1508999585,40cyclictest2539-21chrt23:01:140
1508999585,38cyclictest20746-21/usr/sbin/munin20:55:260
1508999581,33cyclictest23243-21fschecks_time00:00:240
1509099573,39cyclictest30706-21taskset21:20:291
1509099571,43cyclictest21335-21diskstats22:25:231
1509099571,39cyclictest6712-21apt-get23:15:181
1509099571,35cyclictest29552-21latency_hist21:20:061
1508999579,28cyclictest19668-21fschecks_time23:50:250
1508999573,42cyclictest8587-21diskstats21:50:230
1508999571,30cyclictest18231-21unixbench_multi20:45:350
1509099561,43cyclictest28477-21apt-get00:15:211
1509099561,38cyclictest27721-21cron21:15:061
1508999567,5cyclictest23702-21unixbench_singl21:00:350
1508999564,27cyclictest0-21swapper/022:55:060
1508999563,45cyclictest0-21swapper/000:05:180
15089995625,6cyclictest26231-21wget21:10:210
15092995526,15cyclictest11044-21switchtime21:55:313
1509099552,49cyclictest31531-21/usr/sbin/munin19:55:221
1509099551,51cyclictest3095-21apt-get23:05:181
1509099551,50cyclictest26045-21wget19:40:191
1509099551,42cyclictest18794-21apt-get19:20:181
1509099551,41cyclictest0-21swapper/121:55:281
1509099551,40cyclictest2496-21taskset21:33:451
1509099551,38cyclictest15263-21chrt23:37:091
1509099551,33cyclictest9450irq/48-DMA10:44:551
1508999556,39cyclictest0-21swapper/019:59:390
1508999551,32cyclictest13651-21aten_rbpower_vo20:35:220
15089995513,23cyclictest19123-21fschecks_time19:20:240
1509099541,49cyclictest30315-21wget00:20:211
1509099541,49cyclictest30315-21wget00:20:211
1509099541,48cyclictest9037-21latency23:20:251
1509099541,43cyclictest1124-21snmpd23:04:191
1509099541,42cyclictest11404-21taskset20:29:151
1508999547,35cyclictest0-21swapper/022:14:490
1508999546,28cyclictest0-21swapper/000:15:360
1508999541,31cyclictest13940-21/usr/sbin/munin23:35:230
1508999541,28cyclictest8361-21fschecks_count20:20:230
15089995412,5cyclictest9450irq/48-DMA10:44:550
34782530,2sleep30-21swapper/300:35:243
15092995325,14cyclictest21670-21ls22:25:273
15092995320,16cyclictest15744-21awk23:40:183
15092995320,10cyclictest6712-21apt-get23:15:173
1509099531,49cyclictest15807-21grep22:10:221
1509099531,39cyclictest11773-21resolvconf23:28:461
1508999537,28cyclictest0-21swapper/022:17:140
1508999535,25cyclictest19418-21basename22:20:210
1508999535,25cyclictest19418-21basename22:20:210
1508999532,6cyclictest15519-21chrt22:10:060
1508999531,37cyclictest21310-21/usr/sbin/munin19:25:310
1508999531,25cyclictest16014-21fschecks_count23:40:240
15089995310,40cyclictest31576-21taskset00:21:490
15089995310,40cyclictest31576-21taskset00:21:490
1509099521,48cyclictest28338-21apt-get22:45:181
1509099521,43cyclictest25061-21ntp_states21:05:311
1509099521,40cyclictest22668-21taskset23:59:401
1509099521,37cyclictest18523-21seq19:20:011
1509099521,37cyclictest18523-21seq19:20:011
1508999525,7cyclictest7553-21tail21:45:330
1508999524,38cyclictest0-21swapper/022:36:210
1508999521,6cyclictest21472-21fschecks_time23:55:230
1508999521,6cyclictest0-21swapper/023:25:200
15089995213,4cyclictest101ktimersoftd/022:55:360
56292510,2sleep30-21swapper/321:40:353
1509099511,47cyclictest2862-21apt-get21:35:201
1509099511,47cyclictest22916-21apt-get22:30:181
1509099511,47cyclictest13902-21aten_rbpower_cu22:05:211
1508999519,18cyclictest27892-21aten_rbpower_po19:45:200
1508999516,29cyclictest0-21swapper/022:33:580
1508999516,27cyclictest11384-21chrt20:28:590
15089995129,6cyclictest7950irq/56-dwc_otg_00:35:240
1508999512,25cyclictest4759-21taskset20:10:270
15089995122,5cyclictest15796-21ntp_kernel_pll_19:10:280
15092995024,12cyclictest17724-21cat20:45:283
15092995020,7cyclictest24763-21apt-get00:05:183
1509099503,37cyclictest7950irq/56-dwc_otg_20:21:221
1509099501,46cyclictest28829-21timerwakeupswit19:45:341
1509099501,46cyclictest26533-21apt-get22:40:171
1509099501,45cyclictest726-21unixbench_multi00:25:361
1509099501,39cyclictest18822-21sshd22:17:141
1509099501,35cyclictest0-21swapper/122:39:331
1509099501,33cyclictest23521-21memory00:00:291
15090995012,35cyclictest19449-21aten_rbpower_vo23:50:221
1508999501,7cyclictest24691-21fschecks_time21:05:250
1509299493,5cyclictest7750irq/56-dwc_otg22:05:193
15092994925,12cyclictest9350-21/usr/sbin/munin23:20:313
15092994924,11cyclictest30569-21/usr/sbin/munin21:20:273
15092994923,14cyclictest19514-21/usr/sbin/munin20:50:283
15092994920,12cyclictest29665-21latency_hist21:20:073
15092994917,18cyclictest7750irq/56-dwc_otg22:20:243
15092994917,18cyclictest7750irq/56-dwc_otg22:20:243
1509099491,45cyclictest4898-21apt-get23:10:201
1509099491,45cyclictest4898-21apt-get23:10:201
1509099491,38cyclictest0-21swapper/121:41:481
1508999496,26cyclictest27266-21chrt19:41:220
15089994925,12cyclictest12454-21fschecks_time23:30:230
1508999490,30cyclictest0-21swapper/019:36:070
218182480,10sleep2301ktimersoftd/222:25:302
15092994822,10cyclictest2903-21df_inode20:05:203
15092994820,13cyclictest23212-21sensors_temp19:30:313
15092994819,8cyclictest18794-21apt-get19:20:203
15092994813,18cyclictest7950irq/56-dwc_otg_20:42:303
1509099486,31cyclictest7950irq/56-dwc_otg_23:40:191
1509099481,44cyclictest3250-21apt-get00:35:201
1509099481,44cyclictest11980-21apt-get22:00:181
1509099481,43cyclictest32019-21aten_rbpower_po22:55:221
1509099481,43cyclictest15816-21ntp_states19:10:291
1508999485,29cyclictest0-21swapper/020:02:190
15089994817,15cyclictest1208-21idleruntime-cro00:30:050
15092994722,15cyclictest32011-21latency_hist00:25:073
15092994722,15cyclictest32011-21latency_hist00:25:073
15092994722,12cyclictest13022-21sshd20:31:573
15092994722,12cyclictest13022-21sshd20:31:573
15092994719,10cyclictest7950irq/56-dwc_otg_22:45:063
15092994715,12cyclictest0-21swapper/321:25:363
1509099472,42cyclictest15424-21aten_rbpower_po20:40:211
1509099472,41cyclictest6463-21apt-get21:45:181
1509099471,43cyclictest776-21apt-get20:00:151
1509099471,43cyclictest20759-21apt-get20:55:201
1509099471,43cyclictest13626-21aten_rbpower_vo20:35:221
15092994635,8cyclictest22558-21apt-get21:00:183
15092994621,11cyclictest4502-21/usr/sbin/munin20:10:353
15092994616,10cyclictest7950irq/56-dwc_otg_19:25:213
15092994614,18cyclictest2753-21taskset23:04:023
1509199467,25cyclictest0-21swapper/223:55:192
1508999460,31cyclictest6216-21chrt23:12:090
1508999460,31cyclictest6216-21chrt23:12:090
15092994524,9cyclictest17774-21diskstats23:45:263
15092994519,16cyclictest29715-21sshd00:16:143
15092994517,12cyclictest27482-21/usr/sbin/munin00:10:333
15092994516,9cyclictest7850irq/56-dwc_otg_20:02:423
15092994513,12cyclictest7850irq/56-dwc_otg_21:15:063
1509199457,24cyclictest0-21swapper/221:00:202
1509199457,24cyclictest0-21swapper/200:00:202
1509199457,23cyclictest0-21swapper/221:50:112
1509099451,41cyclictest24206-21apt-get19:35:181
1508999448,22cyclictest0-21swapper/021:41:510
1508999447,21cyclictest0-21swapper/022:41:500
15092994319,12cyclictest4400-21seq23:07:023
15092994319,11cyclictest22185-21taskset20:58:353
15092994316,11cyclictest25635-21chrt21:06:383
15092994316,10cyclictest7950irq/56-dwc_otg_00:05:073
15092994314,11cyclictest15255-21latency_hist20:40:063
15092994314,10cyclictest7950irq/56-dwc_otg_19:20:063
15092994314,10cyclictest7950irq/56-dwc_otg_19:20:063
1509199433,14cyclictest29-21rcuc/221:55:272
15091994310,18cyclictest7750irq/56-dwc_otg19:30:352
1508999430,31cyclictest0-21swapper/019:17:210
1508999430,31cyclictest0-21swapper/019:17:210
15092994228,11cyclictest16353-21/usr/sbin/munin22:10:293
15092994220,10cyclictest7950irq/56-dwc_otg_21:50:063
15092994218,11cyclictest1434-21/usr/sbin/munin00:30:253
15092994217,10cyclictest24350-21chrt22:33:403
15092994213,10cyclictest7950irq/56-dwc_otg_23:15:063
15092994213,10cyclictest7950irq/56-dwc_otg_23:15:063
1509199427,22cyclictest0-21swapper/220:35:212
1509199427,21cyclictest0-21swapper/223:15:192
1509199426,25cyclictest0-21swapper/220:25:192
1509199426,22cyclictest0-21swapper/223:05:192
15091994221,13cyclictest301ktimersoftd/220:23:432
15092994127,12cyclictest12627-21latency23:30:263
15092994126,11cyclictest452-21ls22:55:343
15092994126,11cyclictest12173-21/usr/sbin/munin22:00:223
15092994125,13cyclictest1754-21/usr/sbin/munin21:30:293
15092994125,12cyclictest21240-21wget23:55:203
15092994117,11cyclictest7850irq/56-dwc_otg_19:15:053
15092994116,12cyclictest7222-21/usr/sbin/munin20:15:343
15092994113,18cyclictest7950irq/56-dwc_otg_19:54:263
1509299411,24cyclictest0-21swapper/321:35:203
1509199417,22cyclictest0-21swapper/222:35:202
1509199417,22cyclictest0-21swapper/222:15:232
1509199416,26cyclictest0-21swapper/200:05:202
15091994120,13cyclictest31-21ksoftirqd/219:25:282
15091994118,6cyclictest31-21ksoftirqd/221:25:252
15092994026,11cyclictest25126-21if_eth022:35:263
15092994025,12cyclictest27912-21aten_rbpower_vo19:45:213
15092994025,11cyclictest24665-21/usr/sbin/munin19:35:253
15092994016,11cyclictest7950irq/56-dwc_otg_20:00:063
15092994015,6cyclictest18832-21sshd22:17:203
1509199408,13cyclictest31-21ksoftirqd/222:50:282
15091994018,11cyclictest31-21ksoftirqd/200:30:232
1509199401,7cyclictest7950irq/56-dwc_otg_00:25:202
15092993925,11cyclictest28549-21/usr/sbin/munin22:45:223
15092993924,12cyclictest26325-21fschecks_time19:40:223
15092993923,12cyclictest14031-21hostname23:35:233
1509199391,22cyclictest11893-21ls20:30:192
1509199391,22cyclictest11893-21ls20:30:192
15092993825,11cyclictest32122-21wget00:25:223
15092993824,11cyclictest8757-21/usr/sbin/munin20:20:273
15092993824,11cyclictest10230-21fschecks_time20:25:233
15092993823,12cyclictest19593-21fschecks_count23:50:253
15092993823,12cyclictest11031-21ntp_states23:25:293
1509199384,13cyclictest29-21rcuc/222:10:222
15091993823,4cyclictest5585-21ls21:40:352
1509199382,16cyclictest29-21rcuc/223:37:012
1509199381,23cyclictest20995-21fschecks_count20:55:252
15091993715,15cyclictest19204-21if_eth019:20:252
15091993713,4cyclictest12225-21seq23:30:202
1509199362,14cyclictest31-21ksoftirqd/221:20:072
15091993619,7cyclictest31-21ksoftirqd/220:10:332
1509199359,5cyclictest19211-21fschecks_count20:50:242
1509199358,19cyclictest26002-21/usr/sbin/munin19:40:312
1509199355,15cyclictest29-21rcuc/223:01:302
1509199353,14cyclictest31-21ksoftirqd/220:45:222
15091993526,5cyclictest301ktimersoftd/222:55:192
1509199352,3cyclictest0-21swapper/220:00:232
15091993513,7cyclictest301ktimersoftd/200:10:362
1509199351,25cyclictest0-21swapper/219:15:332
1509199351,25cyclictest0-21swapper/219:15:332
15091993512,4cyclictest7750irq/56-dwc_otg20:15:372
1509199348,15cyclictest31-21ksoftirqd/222:06:222
1509199345,18cyclictest301ktimersoftd/223:45:312
1509199344,6cyclictest29-21rcuc/220:45:062
15091993423,4cyclictest7950irq/56-dwc_otg_22:20:302
15091993423,4cyclictest7950irq/56-dwc_otg_22:20:302
1509199338,6cyclictest7850irq/56-dwc_otg_19:50:342
1509199331,15cyclictest0-21swapper/200:35:222
15091993310,9cyclictest301ktimersoftd/221:15:062
1509199327,15cyclictest301ktimersoftd/221:50:062
1509199326,14cyclictest0-21swapper/223:50:212
1509199324,17cyclictest31418-21/usr/sbin/munin19:55:212
1509199323,17cyclictest0-21swapper/219:35:192
1509199321,6cyclictest30027-21cron00:20:062
1509199321,23cyclictest5500-21ls23:10:292
1509199321,23cyclictest5500-21ls23:10:292
1509199321,22cyclictest0-21swapper/223:44:512
1509199321,22cyclictest0-21swapper/222:45:312
1509199316,12cyclictest31-21ksoftirqd/222:30:592
1509199315,14cyclictest2849-21/usr/sbin/munin21:35:202
1509199314,7cyclictest31-21ksoftirqd/223:20:222
1509199312,5cyclictest7850irq/56-dwc_otg_22:40:172
1509199312,15cyclictest0-21swapper/219:45:332
15091993116,9cyclictest301ktimersoftd/200:25:072
15091993116,9cyclictest301ktimersoftd/200:25:072
1509199306,21cyclictest0-21swapper/222:00:192
1509199302,19cyclictest0-21swapper/221:05:272
15091993016,8cyclictest301ktimersoftd/220:05:202
146272302,15sleep10-21swapper/119:07:331
1509199295,14cyclictest7850irq/56-dwc_otg_23:25:252
1509199291,5cyclictest7750irq/56-dwc_otg21:30:232
132912274,14sleep30-21swapper/319:05:213
146682263,14sleep20-21swapper/219:08:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional