You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 15:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 18, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2950125550,3sleep20-21swapper/216:26:102
30544991258,25cyclictest11-21rcu_preempt20:45:581
30543991157,15cyclictest0-21swapper/021:35:580
548521070,27sleep03054399cyclictest20:20:560
157452880,2sleep10-21swapper/120:32:241
143252880,2sleep00-21swapper/019:14:130
160572840,2sleep23054599cyclictest19:16:042
326092800,1sleep30-21swapper/321:31:283
283312780,2sleep20-21swapper/218:51:482
71392750,2sleep17137-21ssh19:06:091
275252740,2sleep20-21swapper/219:29:572
172472740,4sleep2301ktimersoftd/221:52:242
30543996716,36cyclictest9-21ksoftirqd/021:11:030
3054399671,54cyclictest23360-21sh20:41:130
257472670,1sleep10-21swapper/119:26:501
30543996414,26cyclictest24444-21fschecks_time19:26:010
3054399635,33cyclictest9-21ksoftirqd/019:15:590
30543996317,25cyclictest9-21ksoftirqd/020:00:450
184072620,2sleep30-21swapper/320:36:043
3054399615,17cyclictest7950irq/56-dwc_otg_17:06:000
3054399611,50cyclictest13-21rcuc/020:26:020
199212610,10sleep1221ktimersoftd/119:58:551
30543996035,5cyclictest7950irq/56-dwc_otg_18:51:100
30543996010,29cyclictest9-21ksoftirqd/017:11:130
3054399596,42cyclictest0-21swapper/019:45:580
3054499582,9cyclictest0-21swapper/118:15:591
3054399582,34cyclictest32426-21unixbench_singl21:31:140
3054399581,32cyclictest15204-21unixbench_singl18:36:150
3054399563,31cyclictest0-21swapper/017:56:080
30543995631,5cyclictest7850irq/56-dwc_otg_18:30:450
30543995631,5cyclictest7850irq/56-dwc_otg_18:30:450
3054399562,43cyclictest0-21swapper/020:50:460
3054399561,28cyclictest31653-21grep16:31:130
3054399561,28cyclictest31653-21grep16:31:130
30543995611,23cyclictest13570-21munin-run18:35:450
278642560,10sleep0101ktimersoftd/020:08:070
95002550,4sleep339-21ksoftirqd/319:46:083
3054399551,40cyclictest0-21swapper/019:51:160
3054399551,36cyclictest3417-21ls16:45:450
3054399551,33cyclictest3314-21unixbench_multi18:11:130
87332540,2sleep20-21swapper/219:06:542
3054399545,25cyclictest0-21swapper/017:30:590
3054399544,30cyclictest27156-21latency_hist17:50:460
30543995414,25cyclictest9-21ksoftirqd/020:00:470
3054699537,8cyclictest0-21swapper/320:10:573
3054399536,23cyclictest0-21swapper/017:16:020
3054399533,30cyclictest0-21swapper/021:45:460
3054399533,27cyclictest0-21swapper/021:06:000
30543995331,11cyclictest101ktimersoftd/021:50:450
30543995330,5cyclictest7850irq/56-dwc_otg_20:36:110
3054399532,6cyclictest5475-21aten_rbpower_vo16:51:010
3054399531,45cyclictest0-21swapper/018:46:090
3054399531,44cyclictest9659-21wc17:01:070
30543995314,28cyclictest28822-21if_wlan019:31:040
163742530,2sleep30-21swapper/321:12:163
30543995235,6cyclictest7850irq/56-dwc_otg_20:16:150
30543995235,6cyclictest7850irq/56-dwc_otg_20:16:150
3054399522,29cyclictest0-21swapper/019:41:090
3054399522,29cyclictest0-21swapper/019:41:090
3054399522,28cyclictest22739-21/usr/sbin/munin21:21:100
30543995125,5cyclictest19384-21unixbench-2d18:41:140
30543995125,5cyclictest19384-21unixbench-2d18:41:140
3054399512,39cyclictest0-21swapper/020:12:280
3054399511,8cyclictest7416-21latency_hist18:25:450
30543995114,19cyclictest31415-21/usr/sbin/munin18:56:050
3054399511,19cyclictest10011-21latency_hist21:05:460
3054399511,12cyclictest11016-21latency_hist19:10:460
30543995029,4cyclictest7950irq/56-dwc_otg_21:16:080
30543995020,3cyclictest0-21swapper/021:26:150
3054399501,26cyclictest14062-21fschecks_count20:31:020
3054399501,15cyclictest10939-21kworker/u8:220:59:350
3054499497,21cyclictest0-21swapper/117:06:001
3054399494,25cyclictest470-21wc16:36:070
3054399492,27cyclictest0-21swapper/017:41:060
30543994915,18cyclictest3307-21uname19:01:050
193852490,5sleep13054499cyclictest21:55:541
3054499487,26cyclictest30980-21ssh18:56:001
30544994835,9cyclictest221ktimersoftd/121:45:561
3054399488,4cyclictest9-21ksoftirqd/017:51:150
30543994828,5cyclictest7950irq/56-dwc_otg_18:16:060
3054399482,42cyclictest19806-21ssh19:20:560
30543994813,28cyclictest101ktimersoftd/019:40:370
3054599477,25cyclictest0-21swapper/220:00:522
3054499475,11cyclictest0-21swapper/119:00:561
3054399479,21cyclictest13-21rcuc/016:50:170
30543994728,4cyclictest7950irq/56-dwc_otg_17:26:110
3054499464,8cyclictest7750irq/56-dwc_otg17:10:581
3054499463,13cyclictest7750irq/56-dwc_otg16:50:581
3054399464,18cyclictest7750irq/56-dwc_otg16:55:590
30543994619,5cyclictest101ktimersoftd/021:56:130
3054399461,26cyclictest31151-21df_inode18:00:590
3054699453,7cyclictest7950irq/56-dwc_otg_16:55:593
3054499456,24cyclictest0-21swapper/117:55:511
30544994513,5cyclictest0-21swapper/120:06:131
30543994515,15cyclictest15995-21netstat21:51:070
30543994512,4cyclictest908-21runrttasks17:21:080
301522453,34sleep00-21swapper/016:29:210
3054699447,22cyclictest0-21swapper/318:00:573
3054699446,23cyclictest0-21swapper/317:15:583
3054599447,25cyclictest0-21swapper/217:55:512
30545994424,6cyclictest28369-21aten_rbpower_cu19:30:582
3054399441,23cyclictest30744-21ssh20:51:010
3054699437,22cyclictest0-21swapper/316:51:003
3054699432,9cyclictest7950irq/56-dwc_otg_19:55:573
3054599437,23cyclictest0-21swapper/221:05:592
30544994320,13cyclictest0-21swapper/117:00:511
3054399437,18cyclictest0-21swapper/018:06:090
3054699427,21cyclictest0-21swapper/319:20:593
30545994215,18cyclictest0-21swapper/220:11:032
3054599421,33cyclictest0-21swapper/217:40:572
30546994110,5cyclictest7950irq/56-dwc_otg_19:16:103
3054699410,14cyclictest26209-21apt-get20:45:563
3054599416,22cyclictest0-21swapper/217:06:002
196912410,5sleep13054499cyclictest19:20:461
106642410,2sleep110660-21ssh20:26:121
30546994015,3cyclictest381ktimersoftd/318:46:073
3054599401,23cyclictest0-21swapper/217:15:562
30544994010,6cyclictest22582-21unixbench_multi20:01:131
3054399401,25cyclictest0-21swapper/017:36:070
3054699397,5cyclictest7950irq/56-dwc_otg_18:15:453
30546993919,7cyclictest7750irq/56-dwc_otg18:51:143
3054599399,5cyclictest31-21ksoftirqd/220:46:112
3054599397,22cyclictest0-21swapper/217:36:002
3054499394,15cyclictest7950irq/56-dwc_otg_19:30:581
3054499392,4cyclictest7950irq/56-dwc_otg_21:10:461
30544993911,5cyclictest0-21swapper/119:45:471
3054699385,4cyclictest7950irq/56-dwc_otg_20:06:043
30546993813,6cyclictest14994-21unixbench_multi20:31:133
30545993816,12cyclictest10166-21latency_hist21:05:462
3054599381,26cyclictest4867-21ls18:16:122
3054499387,22cyclictest0-21swapper/119:11:011
3054499386,26cyclictest15311-21ssh21:50:591
3054499384,13cyclictest0-21swapper/120:25:461
30544993817,4cyclictest3784-21sh19:40:131
3054699375,6cyclictest7750irq/56-dwc_otg19:35:453
30546993718,7cyclictest7750irq/56-dwc_otg17:06:013
30546993718,7cyclictest381ktimersoftd/317:41:053
30546993716,3cyclictest0-21swapper/318:55:573
3054699371,14cyclictest7950irq/56-dwc_otg_19:41:223
3054699371,14cyclictest7950irq/56-dwc_otg_19:41:223
30546993710,9cyclictest7950irq/56-dwc_otg_21:05:573
3054599377,16cyclictest31-21ksoftirqd/221:11:132
3054599374,24cyclictest0-21swapper/220:20:582
30545993729,6cyclictest31-21ksoftirqd/216:30:562
30545993729,6cyclictest31-21ksoftirqd/216:30:562
3054499378,4cyclictest0-21swapper/121:30:501
3054499374,4cyclictest23867-21fschecks_count17:41:011
3054699368,17cyclictest37-21rcuc/317:54:263
3054699367,15cyclictest0-21swapper/318:41:143
3054699367,15cyclictest0-21swapper/318:41:143
3054699363,23cyclictest0-21swapper/319:35:593
3054699362,18cyclictest7950irq/56-dwc_otg_21:56:043
3054699361,7cyclictest0-21swapper/320:16:003
3054699361,7cyclictest0-21swapper/320:16:003
30546993615,4cyclictest7750irq/56-dwc_otg18:18:273
30546993610,9cyclictest7950irq/56-dwc_otg_20:00:583
3054599362,21cyclictest0-21swapper/221:47:512
3054599362,20cyclictest7950irq/56-dwc_otg_21:16:132
30545993618,11cyclictest10904-21latency_hist21:45:452
30545993614,15cyclictest0-21swapper/220:26:022
3054599361,28cyclictest5902-21ls21:00:452
3054599361,24cyclictest23862-21timerwakeupswit21:21:122
3054599361,24cyclictest1696-21uniq20:16:042
3054599361,24cyclictest1696-21uniq20:16:042
3054499369,7cyclictest18236-21ssh20:36:031
3054499365,14cyclictest7950irq/56-dwc_otg_21:16:021
3054499363,7cyclictest1465-21unixbench_singl18:06:141
3054499361,7cyclictest917-21snmpd21:44:321
229842361,4sleep123072-21cat20:41:111
3054699358,4cyclictest0-21swapper/317:36:133
3054699357,15cyclictest7950irq/56-dwc_otg_18:37:333
3054699356,17cyclictest381ktimersoftd/319:02:183
3054699355,5cyclictest7950irq/56-dwc_otg_20:25:583
3054699354,6cyclictest37-21rcuc/318:34:453
3054699353,5cyclictest7750irq/56-dwc_otg21:45:563
30546993516,6cyclictest7750irq/56-dwc_otg20:54:163
3054599352,12cyclictest0-21swapper/218:59:042
30545993517,4cyclictest29446-21latency_hist20:10:462
30545993512,5cyclictest6847-21timerwakeupswit18:21:132
3054599351,23cyclictest8745-21ls17:00:452
30544993513,6cyclictest26734-21aten_rbpower_en18:50:591
3054499351,12cyclictest0-21swapper/120:51:121
30544993510,6cyclictest0-21swapper/119:42:551
3054699349,8cyclictest7950irq/56-dwc_otg_17:35:463
3054699343,6cyclictest39-21ksoftirqd/318:30:453
3054699343,6cyclictest39-21ksoftirqd/318:30:453
3054699343,10cyclictest15215-21ssh19:52:543
30546993419,5cyclictest11449-21cpu19:11:023
30546993414,5cyclictest908-21runrttasks21:17:173
30546993410,9cyclictest7950irq/56-dwc_otg_21:00:003
3054599349,15cyclictest31-21ksoftirqd/218:46:012
3054599349,12cyclictest22463-21tune2fs20:41:022
3054599349,10cyclictest32613-21ssh19:35:572
3054599342,20cyclictest7750irq/56-dwc_otg20:33:132
3054599341,4cyclictest0-21swapper/219:20:562
3054599341,24cyclictest0-21swapper/221:57:552
3054499347,15cyclictest0-21swapper/119:15:581
3054499346,4cyclictest7750irq/56-dwc_otg17:51:111
3054499345,7cyclictest7950irq/56-dwc_otg_21:21:061
3054499343,5cyclictest0-21swapper/120:11:141
3054499343,3cyclictest7950irq/56-dwc_otg_18:50:451
3054499342,6cyclictest7850irq/56-dwc_otg_16:30:561
3054499342,6cyclictest7850irq/56-dwc_otg_16:30:561
3054499342,3cyclictest7950irq/56-dwc_otg_18:10:561
30544993413,4cyclictest10188-21sed18:31:071
3054699339,11cyclictest0-21swapper/319:26:013
3054699336,13cyclictest7950irq/56-dwc_otg_18:21:073
3054699331,3cyclictest7750irq/56-dwc_otg17:20:573
3054699331,21cyclictest26443-21ls17:46:123
30546993311,12cyclictest7950irq/56-dwc_otg_21:25:173
3054599338,15cyclictest5676-21grep19:41:132
3054599338,15cyclictest5676-21grep19:41:132
3054599336,18cyclictest0-21swapper/217:25:092
3054599333,5cyclictest9450irq/48-DMA09:12:252
3054599332,10cyclictest0-21swapper/218:38:022
3054599331,7cyclictest28786-21munin-run17:55:452
30545993316,15cyclictest522150ssh21:38:072
3054599331,13cyclictest3296-21tail18:11:132
30545993310,11cyclictest32451-21unixbench_singl21:31:142
3054499339,12cyclictest0-21swapper/121:26:111
3054499338,15cyclictest0-21swapper/117:45:591
3054499338,14cyclictest0-21swapper/117:26:061
30544993319,5cyclictest14384-21sh18:36:051
30544993318,4cyclictest10586-21sh19:47:111
3054499331,22cyclictest4148-21missed_timers16:46:081
3054699328,11cyclictest29415-21ssh21:28:243
3054699327,9cyclictest37-21rcuc/316:36:393
3054699325,14cyclictest39-21ksoftirqd/319:07:523
3054699323,4cyclictest7750irq/56-dwc_otg20:21:033
30546993218,5cyclictest1702-21seq18:08:553
3054699321,15cyclictest381ktimersoftd/321:44:483
3054699321,13cyclictest7950irq/56-dwc_otg_20:43:063
3054599329,9cyclictest31-21ksoftirqd/217:46:032
3054599329,12cyclictest0-21swapper/221:26:422
3054599326,7cyclictest7850irq/56-dwc_otg_18:30:452
3054599326,7cyclictest7850irq/56-dwc_otg_18:30:452
30545993212,10cyclictest8842-21cut19:45:592
3054599321,14cyclictest0-21swapper/220:40:462
3054499329,5cyclictest21-21rcuc/120:57:411
3054499327,15cyclictest0-21swapper/121:35:581
3054499325,6cyclictest7750irq/56-dwc_otg18:26:141
3054499325,6cyclictest7750irq/56-dwc_otg18:26:141
3054499322,4cyclictest7750irq/56-dwc_otg17:20:461
3054499322,3cyclictest0-21swapper/117:21:141
3054499321,20cyclictest14672-21ssh21:11:001
300922324,20sleep30-21swapper/316:28:353
169152320,2sleep231-21ksoftirqd/219:55:462
3054699316,13cyclictest381ktimersoftd/317:01:013
3054699312,19cyclictest0-21swapper/316:41:253
3054699312,14cyclictest37-21rcuc/321:36:543
3054699311,12cyclictest7750irq/56-dwc_otg16:31:073
3054699311,12cyclictest7750irq/56-dwc_otg16:31:073
3054699311,11cyclictest7950irq/56-dwc_otg_21:05:413
30546993110,10cyclictest7750irq/56-dwc_otg21:51:243
3054599318,13cyclictest0-21swapper/217:26:092
3054599318,12cyclictest32355-21/usr/sbin/munin16:35:572
3054599317,10cyclictest0-21swapper/218:35:452
3054599313,4cyclictest7950irq/56-dwc_otg_19:55:592
30545993125,4cyclictest13722-21unixbench-2d17:11:142
3054599311,21cyclictest0-21swapper/218:04:332
30545993110,4cyclictest0-21swapper/220:51:002
3054499319,4cyclictest0-21swapper/118:41:491
3054499319,4cyclictest0-21swapper/118:41:491
3054499318,4cyclictest0-21swapper/120:16:561
3054499318,4cyclictest0-21swapper/120:16:561
3054499316,15cyclictest0-21swapper/117:36:111
3054499312,5cyclictest32502-21munin-run18:05:451
3054499311,15cyclictest7950irq/56-dwc_otg_16:56:031
3054699305,17cyclictest7850irq/56-dwc_otg_17:25:513
3054699301,15cyclictest0-21swapper/317:55:503
3054599309,14cyclictest0-21swapper/219:01:102
3054599304,18cyclictest7950irq/56-dwc_otg_18:41:012
3054599304,18cyclictest7950irq/56-dwc_otg_18:41:012
3054499306,4cyclictest7950irq/56-dwc_otg_21:01:111
3054499301,3cyclictest0-21swapper/116:35:591
3054699296,14cyclictest0-21swapper/316:46:003
3054599297,14cyclictest1345-21ls18:06:132
3054599292,8cyclictest21559-21apt17:35:452
30545992918,4cyclictest1735-21/usr/sbin/munin16:40:592
30545992913,4cyclictest301ktimersoftd/219:10:472
30544992923,4cyclictest1948-21ls16:41:011
30544992910,4cyclictest13774-21ssh19:51:091
3054599286,15cyclictest7750irq/56-dwc_otg16:51:072
3054599281,18cyclictest0-21swapper/216:46:052
3054499281,17cyclictest0-21swapper/117:31:121
301832283,15sleep10-21swapper/116:29:471
3054699272,6cyclictest0-21swapper/317:11:083
3054499271,13cyclictest7750irq/56-dwc_otg18:23:531
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional