You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-21 - 01:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 20, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2214222230,2sleep20-21swapper/216:26:162
657321000,5sleep32277399cyclictest20:32:363
33102980,1sleep10-21swapper/118:32:001
272292890,9sleep1221ktimersoftd/116:41:111
83322800,2sleep20-21swapper/219:16:142
174012800,2sleep217403-21rm21:25:492
84592770,4sleep38426-21ssh19:16:153
287102770,3sleep028708-21ssh20:20:570
29862760,2sleep32277399cyclictest19:49:483
29862760,2sleep32277399cyclictest19:49:483
232692710,1sleep20-21swapper/220:52:562
131072710,2sleep20-21swapper/218:44:052
37062690,1sleep30-21swapper/319:11:073
2277099682,7cyclictest7323-21ssh19:16:000
2277199661,6cyclictest25727-21latency_hist21:35:471
22770996429,16cyclictest31056-21idleruntime-cro19:05:460
2277099631,48cyclictest2235-21grep18:31:090
2277099629,15cyclictest13-21rcuc/018:35:580
2277099622,40cyclictest25237-21ssh20:56:000
202562620,2sleep0101ktimersoftd/021:28:530
2277199616,48cyclictest0-21swapper/118:20:571
2277099617,18cyclictest0-21swapper/018:20:570
2277099601,37cyclictest0-21swapper/019:06:010
2277099585,10cyclictest24601-21sshd20:15:590
2277099580,41cyclictest5452-21kworker/0:220:51:020
283542570,4sleep028341-21ssh19:41:040
2277099576,28cyclictest10508-21fschecks_time18:41:040
77112560,5sleep22277299cyclictest20:34:402
22770995615,23cyclictest0-21swapper/021:56:010
22770995615,23cyclictest0-21swapper/021:56:010
2277099561,32cyclictest0-21swapper/021:11:080
2277099551,30cyclictest30234-21ls16:51:020
22770995436,4cyclictest0-21swapper/021:45:470
22770995436,4cyclictest0-21swapper/021:45:470
2277299534,27cyclictest7850irq/56-dwc_otg_20:20:562
22770995335,5cyclictest0-21swapper/021:36:000
2277099532,29cyclictest27641-21grep18:11:140
22770995318,5cyclictest2000-21fschecks_time21:46:040
22770995318,20cyclictest20965-21cron17:55:460
2277299521,12cyclictest23642-21ls19:35:462
22770995229,9cyclictest0-21swapper/020:06:090
130402520,2sleep213012-21ssh21:20:462
2277099512,6cyclictest29282-21apt-get21:00:590
2277099512,27cyclictest0-21swapper/018:46:010
2277099511,41cyclictest0-21swapper/019:49:260
2277099511,41cyclictest0-21swapper/019:49:260
2277099511,31cyclictest6674-21grep21:51:120
2277099511,26cyclictest4117-21grep19:51:010
2277099511,22cyclictest2092550ssh20:11:050
2277299501,20cyclictest0-21swapper/219:20:522
2277099503,30cyclictest0-21swapper/019:40:170
22770995027,5cyclictest29589-21unixbench_singl18:16:150
22770995027,3cyclictest0-21swapper/021:06:000
22770995027,3cyclictest0-21swapper/021:06:000
2277099502,29cyclictest21676-21apt-get21:30:530
2277099501,21cyclictest0-21swapper/020:41:120
269582490,5sleep32277399cyclictest20:57:083
2277099492,26cyclictest32438-21missed_timers16:56:080
2277099491,29cyclictest8857-21ssh19:56:140
2277099491,18cyclictest0-21swapper/017:25:460
22773994829,9cyclictest1832350ssh18:50:463
22770994822,13cyclictest0-21swapper/020:49:260
2277099481,25cyclictest9937-21diskmemload20:36:160
2277099481,25cyclictest9937-21diskmemload20:36:160
2277099474,5cyclictest0-21swapper/019:30:580
2277099472,8cyclictest13316-21unixbench_multi17:31:140
22770994725,3cyclictest0-21swapper/018:51:000
22770994725,3cyclictest0-21swapper/018:51:000
22770994723,5cyclictest1335-21wget17:01:020
2277099472,26cyclictest0-21swapper/017:16:050
22770994722,12cyclictest3261-21aten_rbpower_en19:11:010
22770994719,18cyclictest7750irq/56-dwc_otg21:21:170
2277099471,28cyclictest14740-21ps17:36:110
2277099471,27cyclictest17707-21cut17:46:000
22773994624,12cyclictest22759-21ssh20:13:033
2277099464,29cyclictest12783-21unixbench_singl19:21:140
2277099462,23cyclictest0-21swapper/020:33:310
22770994621,2cyclictest0-21swapper/018:56:000
22770994617,5cyclictest4629-21basename17:10:460
2277099461,28cyclictest26540-21wget16:41:000
2277099461,27cyclictest25353-21wc16:36:100
2277099461,23cyclictest13005-21ssh20:01:120
2277299451,24cyclictest7850irq/56-dwc_otg_20:40:592
2277199457,24cyclictest0-21swapper/119:00:521
2277199453,16cyclictest10225-21ls21:16:131
22770994522,4cyclictest345-21proc_pri18:26:120
2277099452,24cyclictest0-21swapper/017:41:050
2277099451,31cyclictest0-21swapper/017:59:040
2277099451,31cyclictest0-21swapper/017:59:040
223662453,35sleep00-21swapper/016:29:020
2277399447,24cyclictest0-21swapper/321:31:003
22773994422,4cyclictest30317-21fschecks_count21:41:033
22773994422,4cyclictest30317-21fschecks_count21:41:033
2277199447,22cyclictest0-21swapper/118:26:001
22771994414,4cyclictest1195-21diskmemload21:25:441
2277099443,22cyclictest0-21swapper/020:30:470
2277099442,6cyclictest6022-21grep17:11:140
2277099442,6cyclictest6022-21grep17:11:140
2277099442,24cyclictest0-21swapper/019:26:140
2277099435,21cyclictest13-21rcuc/016:35:140
2277099433,29cyclictest26603-21kworker/0:021:16:160
2277099432,8cyclictest25921-21grep18:06:150
22770994315,8cyclictest24600-21idleruntime-cro18:05:460
2277299427,24cyclictest0-21swapper/221:36:002
2277199427,26cyclictest7950irq/56-dwc_otg_20:51:011
2277199427,25cyclictest0-21swapper/118:11:001
2277199427,23cyclictest0-21swapper/117:40:591
2277199427,23cyclictest0-21swapper/116:58:301
22771994220,4cyclictest15324-21idleruntime-cro19:25:461
22771994219,8cyclictest0-21swapper/120:46:051
68252411,35sleep16835-21ssh21:12:001
2277299417,24cyclictest0-21swapper/216:58:302
22772994119,4cyclictest7950irq/56-dwc_otg_19:11:072
22771994121,6cyclictest5913-21aten_rbpower_po21:51:011
22770994122,5cyclictest7950irq/56-dwc_otg_16:50:460
22770994114,13cyclictest10377-21/usr/sbin/munin17:26:080
99282400,2sleep19606-21apt-get21:55:481
99282400,2sleep19606-21apt-get21:55:481
2277399406,24cyclictest0-21swapper/321:36:003
22772994026,5cyclictest31522-21ssh21:02:572
2277299402,5cyclictest14436-21ssh18:45:582
22772994019,5cyclictest27543-21sh19:40:302
2277199405,17cyclictest21407-21ssh21:30:471
2277199402,9cyclictest0-21swapper/118:55:591
22771994024,5cyclictest221ktimersoftd/118:36:001
22771994019,12cyclictest908-21runrttasks17:11:111
22771994019,12cyclictest908-21runrttasks17:11:111
22771994015,10cyclictest0-21swapper/119:40:581
147992400,1sleep30-21swapper/319:24:523
2277299393,13cyclictest0-21swapper/218:31:052
2277299392,3cyclictest7950irq/56-dwc_otg_18:10:592
2277299392,26cyclictest7950irq/56-dwc_otg_16:36:072
22772993914,4cyclictest22062-21fschecks_count21:31:052
22771993919,5cyclictest908-21runrttasks19:47:191
22771993919,5cyclictest908-21runrttasks19:47:191
22773993820,9cyclictest19749-21/usr/sbin/munin19:31:093
22773993820,8cyclictest8548-21ssh21:15:093
2277299389,4cyclictest0-21swapper/220:01:012
2277299386,17cyclictest7850irq/56-dwc_otg_17:16:042
22772993818,9cyclictest4388-21sh21:10:052
22772993818,9cyclictest4388-21sh21:10:052
22772993813,6cyclictest11587-21sh21:56:402
22772993813,6cyclictest11587-21sh21:56:402
22771993811,18cyclictest27154-21unixbench_singl21:36:161
2277399379,13cyclictest7950irq/56-dwc_otg_19:01:013
22773993729,6cyclictest17141-21ssh20:06:123
22773993717,6cyclictest27708-21idleruntime-cro19:40:463
22773993716,5cyclictest10550-21taskset20:37:183
22773993716,5cyclictest10550-21taskset20:37:183
2277299377,21cyclictest7850irq/56-dwc_otg_18:36:042
2277299376,21cyclictest7850irq/56-dwc_otg_18:01:152
22772993719,10cyclictest22884-21sh18:55:542
2277299371,7cyclictest32378-21grep18:26:042
22772993717,5cyclictest3532-21sh20:29:342
22772993716,6cyclictest7327-21sh21:51:362
2277199375,15cyclictest7750irq/56-dwc_otg20:31:161
2277199374,4cyclictest7850irq/56-dwc_otg_19:35:471
2277199371,25cyclictest14199-21ls18:45:471
2277199371,12cyclictest0-21swapper/119:56:111
22773993625,8cyclictest21388-21tune2fs20:51:033
2277299366,20cyclictest7850irq/56-dwc_otg_20:16:002
22772993618,9cyclictest10283-21latency_hist17:25:462
2277299361,14cyclictest25577-21systemd-journal21:22:302
2277199366,24cyclictest13980-21/usr/sbin/munin17:36:061
2277199365,22cyclictest14453-21ssh18:46:001
2277199363,17cyclictest13117-21grep20:41:041
22771993624,9cyclictest0-21swapper/120:16:031
2277199361,24cyclictest2665-21ls21:46:131
22771993610,14cyclictest0-21swapper/119:10:551
2277399359,13cyclictest0-21swapper/321:16:123
22773993513,4cyclictest6868-21aten_rbpower_vo17:16:013
2277299358,24cyclictest6168-21ls21:11:132
2277299357,5cyclictest0-21swapper/219:42:482
2277299353,3cyclictest7950irq/56-dwc_otg_19:56:112
2277299352,8cyclictest16579-21tune2fs20:06:032
2277199356,22cyclictest7950irq/56-dwc_otg_20:11:021
22771993529,3cyclictest300242sleep116:50:581
22771993513,6cyclictest908-21runrttasks17:18:061
2277199351,25cyclictest0-21swapper/121:01:021
91112340,2sleep30-21swapper/321:54:533
303872340,3sleep230389-21fschecks_time21:41:032
303872340,3sleep230389-21fschecks_time21:41:032
2277399349,3cyclictest0-21swapper/318:57:193
2277399348,5cyclictest7850irq/56-dwc_otg_20:30:463
22773993413,9cyclictest7950irq/56-dwc_otg_21:56:143
22773993413,9cyclictest7950irq/56-dwc_otg_21:56:143
22773993410,4cyclictest0-21swapper/321:23:143
2277299348,12cyclictest7950irq/56-dwc_otg_21:46:042
2277299342,5cyclictest0-21swapper/216:31:142
22772993417,4cyclictest28716-21sh21:00:222
2277299341,25cyclictest7850irq/56-dwc_otg_17:31:142
22772993412,4cyclictest19364-21latency_hist17:50:472
2277299341,17cyclictest0-21swapper/217:36:062
2277299341,12cyclictest0-21swapper/218:20:562
2277199347,15cyclictest0-21swapper/117:50:591
2277199346,15cyclictest0-21swapper/117:01:011
22771993421,7cyclictest221ktimersoftd/119:06:161
22771993419,4cyclictest32221-21sh20:25:161
22771993415,10cyclictest7950irq/56-dwc_otg_20:06:251
22771993412,11cyclictest0-21swapper/120:57:131
2277199341,18cyclictest7950irq/56-dwc_otg_17:21:141
22752340,3sleep12274-21ssh21:06:161
22752340,3sleep12274-21ssh21:06:161
2277399336,13cyclictest7950irq/56-dwc_otg_19:45:473
2277399335,14cyclictest7750irq/56-dwc_otg19:51:303
2277399334,9cyclictest7950irq/56-dwc_otg_16:41:003
22773993313,5cyclictest25923-21tail18:06:153
22773993310,4cyclictest0-21swapper/318:52:313
22773993310,4cyclictest0-21swapper/318:52:313
2277299337,23cyclictest0-21swapper/220:10:522
2277299336,7cyclictest7850irq/56-dwc_otg_18:16:092
22772993318,3cyclictest7850irq/56-dwc_otg_19:46:052
22772993318,3cyclictest7850irq/56-dwc_otg_19:46:052
22772993314,5cyclictest301ktimersoftd/218:50:512
22772993314,5cyclictest301ktimersoftd/218:50:512
22772993313,12cyclictest0-21swapper/220:36:132
22772993313,12cyclictest0-21swapper/220:36:132
2277299331,22cyclictest0-21swapper/216:40:562
2277199337,4cyclictest7850irq/56-dwc_otg_19:55:011
2277199333,13cyclictest0-21swapper/121:45:221
2277199333,13cyclictest0-21swapper/121:45:221
22771993317,4cyclictest908-21runrttasks19:19:431
22771993316,5cyclictest28709-21fschecks_count18:16:031
2277199331,17cyclictest0-21swapper/118:00:481
22771993310,5cyclictest0-21swapper/120:38:281
22771993310,5cyclictest0-21swapper/120:38:281
2277399326,15cyclictest7850irq/56-dwc_otg_17:36:103
2277399323,15cyclictest7950irq/56-dwc_otg_18:11:133
22773993217,5cyclictest966-21sh21:05:593
22773993217,5cyclictest966-21sh21:05:593
2277399321,4cyclictest7950irq/56-dwc_otg_19:56:163
2277399321,3cyclictest0-21swapper/317:25:463
2277399321,26cyclictest12326-21ls20:01:013
22773993211,11cyclictest22666-21taskset17:59:243
22773993211,11cyclictest22666-21taskset17:59:243
2277299327,11cyclictest7950irq/56-dwc_otg_17:11:132
2277299327,11cyclictest7950irq/56-dwc_otg_17:11:132
2277299324,13cyclictest7950irq/56-dwc_otg_17:41:032
2277299323,12cyclictest30790-21ps16:51:112
2277299321,6cyclictest17178-21/usr/sbin/munin20:46:022
2277299321,5cyclictest4882-21wc19:51:112
2277299321,19cyclictest0-21swapper/219:28:032
2277199327,14cyclictest0-21swapper/120:01:001
2277199327,12cyclictest27707-21latency_hist19:40:471
2277199324,5cyclictest25412-21ls18:06:081
2277199322,11cyclictest0-21swapper/116:36:351
22771993214,4cyclictest19432-21latency_hist19:30:471
122692320,2sleep30-21swapper/318:42:373
2277399317,6cyclictest29051-21perl20:21:013
2277399317,4cyclictest7850irq/56-dwc_otg_18:31:053
2277399317,14cyclictest0-21swapper/317:01:013
2277399316,5cyclictest29196-21proc_pri18:16:113
2277399311,7cyclictest0-21swapper/317:45:583
2277399311,6cyclictest26184-21modprobe16:40:463
22773993112,4cyclictest32551-21cat19:06:163
2277399311,12cyclictest7950irq/56-dwc_otg_17:45:473
2277299317,4cyclictest0-21swapper/219:06:082
2277299317,14cyclictest0-21swapper/217:01:012
2277299316,13cyclictest0-21swapper/219:05:052
2277299312,20cyclictest7850irq/56-dwc_otg_18:06:152
2277299311,20cyclictest0-21swapper/217:50:492
2277199316,4cyclictest0-21swapper/118:52:441
2277199316,4cyclictest0-21swapper/118:52:441
2277199313,6cyclictest7750irq/56-dwc_otg17:56:151
2277199313,6cyclictest7750irq/56-dwc_otg17:56:151
2277199311,15cyclictest612-21ls20:26:021
2277199311,13cyclictest908-21runrttasks17:46:591
185192311,4sleep30-21swapper/320:46:463
2277399307,10cyclictest7950irq/56-dwc_otg_21:00:593
2277399306,5cyclictest7950irq/56-dwc_otg_18:21:123
2277399306,4cyclictest7750irq/56-dwc_otg20:16:003
2277399305,12cyclictest0-21swapper/319:29:553
2277399301,6cyclictest21123-21latency_hist17:55:473
2277399301,4cyclictest0-21swapper/318:01:153
2277299301,15cyclictest0-21swapper/217:08:292
2277199303,13cyclictest7750irq/56-dwc_otg17:31:001
2277199301,21cyclictest908-21runrttasks17:10:041
2277399296,9cyclictest0-21swapper/320:44:013
2277399296,11cyclictest0-21swapper/318:36:443
2277399296,10cyclictest7850irq/56-dwc_otg_21:46:063
2277399295,4cyclictest7850irq/56-dwc_otg_16:58:293
22773992923,4cyclictest4742-21latency_hist17:10:473
2277399291,8cyclictest6060-21grep17:11:143
2277399291,8cyclictest6060-21grep17:11:143
2277399291,8cyclictest31152-21unixbench_multi16:51:153
2277299292,17cyclictest0-21swapper/217:26:062
2277199292,13cyclictest0-21swapper/116:47:541
2277199291,19cyclictest10649-21fschecks_count17:26:041
223922293,19sleep10-21swapper/116:29:241
223162295,15sleep30-21swapper/316:28:243
2277399287,14cyclictest17427-21sh21:25:523
2277299281,15cyclictest7750irq/56-dwc_otg16:46:082
2277299281,13cyclictest21447-21/usr/sbin/munin17:56:042
2277299281,13cyclictest21447-21/usr/sbin/munin17:56:042
22771992818,4cyclictest221ktimersoftd/116:31:121
2277399277,12cyclictest0-21swapper/318:25:563
2277399271,17cyclictest0-21swapper/316:34:513
2277399267,10cyclictest7950irq/56-dwc_otg_17:26:083
2277399265,4cyclictest7950irq/56-dwc_otg_17:31:033
22773992615,3cyclictest7850irq/56-dwc_otg_16:46:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional