You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 22:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Apr 11, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2614421880,5sleep01551399cyclictest19:07:160
1124921190,2sleep011250-21ssh20:06:020
155139910133,15cyclictest0-21swapper/021:20:490
1551399986,84cyclictest0-21swapper/021:30:560
1551399954,53cyclictest0-21swapper/019:50:530
1551499832,45cyclictest30280-21ssh19:50:531
147412820,2sleep30-21swapper/318:54:473
1551399793,67cyclictest18653-21unixbench_multi18:06:100
15513997920,24cyclictest1337-21sh21:10:560
93202780,3sleep315512-21cyclictest19:25:593
1551399778,49cyclictest0-21swapper/020:55:520
1551399776,51cyclictest17995-21unixbench_singl20:51:120
15513997745,9cyclictest6748-21latency_hist18:45:420
1551399762,57cyclictest0-21swapper/021:25:550
1551399762,14cyclictest14961-21sh20:10:540
1551399761,38cyclictest0-21swapper/016:55:540
1551399761,38cyclictest0-21swapper/016:55:540
1551399743,62cyclictest0-21swapper/019:11:030
1551399743,62cyclictest0-21swapper/019:11:030
1551399738,47cyclictest20150-21/usr/sbin/munin19:00:580
15513997313,50cyclictest4904-21aten_rbpower_po17:30:560
192722720,3sleep019274-21aten_rbpower_vo16:40:560
1551399727,54cyclictest0-21swapper/021:40:560
1551399721,58cyclictest2940-21apt21:50:420
1551399721,57cyclictest22327-21grep18:16:090
15513997117,37cyclictest0-21swapper/016:30:440
1551399693,45cyclictest8521-21wget17:40:570
1551399693,45cyclictest8521-21wget17:40:570
1551399692,54cyclictest0-21swapper/020:21:110
1551399692,54cyclictest0-21swapper/020:21:110
217812680,2sleep20-21swapper/221:34:342
1551399681,58cyclictest2095-21/usr/sbin/munin19:56:020
1551399681,58cyclictest2095-21/usr/sbin/munin19:56:020
1551399681,52cyclictest917-21snmpd17:36:110
1551699669,11cyclictest7850irq/56-dwc_otg_20:50:543
1551699665,6cyclictest7950irq/56-dwc_otg_20:25:593
15513996647,5cyclictest15056-21grep17:56:110
15513996643,7cyclictest15706-21aten_rbpower_en18:55:560
1551399661,55cyclictest7219-21modprobe21:55:420
15513996610,40cyclictest101ktimersoftd/020:26:100
15516996520,5cyclictest9999-21latency_hist17:45:423
15516996520,5cyclictest9999-21latency_hist17:45:423
15513996545,5cyclictest30124-21grep17:10:580
1551399652,49cyclictest8662-21latency_hist19:25:420
1551399651,54cyclictest21054-21aten_rbpower_po16:45:570
1551399649,31cyclictest27896-21runrttasks18:31:090
1551399642,56cyclictest0-21swapper/018:00:530
1551399642,47cyclictest0-21swapper/020:49:310
222742630,2sleep10-21swapper/119:02:581
1551399637,44cyclictest4317-21cron19:20:410
15513996342,8cyclictest7750irq/56-dwc_otg20:16:050
1551399631,50cyclictest12860-21apt19:30:420
15513996314,30cyclictest0-21swapper/018:20:550
1551399630,55cyclictest32060-21taskset21:08:210
1551399622,51cyclictest0-21swapper/020:41:020
1551399611,52cyclictest11652-21uniq18:50:590
15513996112,36cyclictest101ktimersoftd/019:37:510
1551399601,50cyclictest0-21swapper/021:38:190
1551399600,20cyclictest23847-21timerwakeupswit16:51:090
1551699599,4cyclictest0-21swapper/319:16:043
1551399596,42cyclictest0-21swapper/020:33:400
15513995945,5cyclictest7850irq/56-dwc_otg_21:01:090
1551399592,41cyclictest0-21swapper/017:01:060
1551399582,45cyclictest0-21swapper/018:36:010
1551399581,45cyclictest1460-21grep17:21:000
1551399581,36cyclictest0-21swapper/017:50:540
1551499572,20cyclictest0-21swapper/116:55:541
1551499572,20cyclictest0-21swapper/116:55:541
15514995710,18cyclictest0-21swapper/120:15:551
1551399576,27cyclictest10308-21kworker/u8:321:55:590
1551399576,27cyclictest10308-21kworker/u8:321:55:590
1551399574,43cyclictest0-21swapper/019:46:010
1551399572,45cyclictest0-21swapper/018:10:530
1551399572,45cyclictest0-21swapper/018:10:530
1551399572,44cyclictest0-21swapper/019:30:590
1551399572,44cyclictest0-21swapper/019:30:590
15513995710,33cyclictest4711-21latency_hist17:30:420
1551399564,42cyclictest7303-21ssh21:16:510
1551399562,46cyclictest0-21swapper/020:01:100
15513995612,31cyclictest101ktimersoftd/018:50:160
15513995610,34cyclictest11468-21grep17:46:080
15515995547,5cyclictest12436-21ssh20:45:552
1551399553,39cyclictest25915-21unixbench_singl18:26:100
1551399551,43cyclictest0-21swapper/020:35:520
1551699546,10cyclictest12196-21ssh20:45:423
1551599548,21cyclictest0-21swapper/220:00:552
1551599547,28cyclictest0-21swapper/217:00:552
1551399541,39cyclictest0-21swapper/019:41:010
1551599534,17cyclictest0-21swapper/216:30:432
284572520,1sleep30-21swapper/318:31:563
283372520,2sleep30-21swapper/319:47:543
1551499527,27cyclictest0-21swapper/117:00:551
15514995215,25cyclictest0-21swapper/121:20:541
154012520,6sleep31551699cyclictest19:32:443
154012520,6sleep31551699cyclictest19:32:443
1551699511,17cyclictest32011-21ls17:15:573
1551499511,27cyclictest0-21swapper/120:10:551
1551599507,29cyclictest0-21swapper/216:50:552
1551499507,18cyclictest10078-21ls17:45:421
1551499507,18cyclictest10078-21ls17:45:421
1551499501,3cyclictest0-21swapper/121:10:561
1551499499,12cyclictest7950irq/56-dwc_otg_16:30:441
1551499497,28cyclictest0-21swapper/118:20:551
1551499496,25cyclictest0-21swapper/117:45:471
151112493,12sleep00-21swapper/016:29:010
1551599487,8cyclictest7950irq/56-dwc_otg_21:20:532
1551599487,28cyclictest0-21swapper/218:05:552
1551599486,6cyclictest7950irq/56-dwc_otg_19:40:522
1551499485,25cyclictest21-21rcuc/120:25:421
1551499485,25cyclictest21-21rcuc/120:25:421
1551399488,30cyclictest870-21chrt17:19:560
1551399482,31cyclictest0-21swapper/017:06:500
15513994823,16cyclictest0-21swapper/016:40:320
15515994723,5cyclictest15319-21fschecks_time20:11:002
1551499478,25cyclictest0-21swapper/119:15:471
1551499473,11cyclictest0-21swapper/119:45:551
15514994724,9cyclictest7726-21unixbench_multi17:36:111
1551599467,25cyclictest0-21swapper/216:45:562
1551499468,26cyclictest0-21swapper/117:50:561
1551499467,25cyclictest0-21swapper/117:20:571
15514994615,8cyclictest0-21swapper/120:05:531
1551599457,24cyclictest0-21swapper/219:15:472
1551599457,24cyclictest0-21swapper/217:50:562
1551599447,24cyclictest0-21swapper/217:25:562
1551599447,23cyclictest0-21swapper/221:35:472
15515994422,5cyclictest16802-21aten_rbpower_po20:50:572
1551499447,5cyclictest0-21swapper/118:30:541
15514994423,8cyclictest221ktimersoftd/119:40:421
254722430,5sleep31551699cyclictest18:26:053
1551599437,6cyclictest7850irq/56-dwc_otg_18:40:552
15515994321,13cyclictest7750irq/56-dwc_otg19:40:382
1551499437,23cyclictest0-21swapper/118:55:551
1551499431,4cyclictest22519-21apt-config21:35:421
15514994314,15cyclictest4856-21aten_rbpower_en17:30:561
163742420,3sleep30-21swapper/318:56:043
1551599427,22cyclictest0-21swapper/218:20:552
15515994215,8cyclictest7950irq/56-dwc_otg_19:09:492
1551499426,23cyclictest0-21swapper/117:15:561
1551499421,22cyclictest0-21swapper/120:00:551
146742427,24sleep314767-21/usr/sbin/munin16:26:083
1551599417,25cyclictest0-21swapper/220:55:542
1551599417,24cyclictest0-21swapper/217:45:542
1551599417,11cyclictest301ktimersoftd/217:40:582
1551599417,11cyclictest301ktimersoftd/217:40:582
15515994118,11cyclictest7850irq/56-dwc_otg_18:31:172
15515994117,8cyclictest7850irq/56-dwc_otg_19:21:052
15515994115,14cyclictest7750irq/56-dwc_otg19:48:582
1551499414,23cyclictest0-21swapper/119:25:271
15514994115,6cyclictest32452-21grep19:15:421
15514994115,6cyclictest32452-21grep19:15:421
47902400,2sleep30-21swapper/318:42:193
4332400,1sleep30-21swapper/318:37:163
316642400,1sleep10-21swapper/121:07:391
1551599408,10cyclictest0-21swapper/221:25:562
15515994015,8cyclictest7750irq/56-dwc_otg20:31:282
1551599401,10cyclictest0-21swapper/217:56:092
1551499406,24cyclictest21-21rcuc/121:56:471
1551499406,24cyclictest21-21rcuc/121:56:471
1551499405,17cyclictest21-21rcuc/121:28:361
1551499403,22cyclictest0-21swapper/119:40:521
1551499403,20cyclictest21-21rcuc/121:45:271
1551499402,8cyclictest0-21swapper/118:35:551
1551499402,11cyclictest0-21swapper/116:45:541
1551599395,23cyclictest15649-21ssh18:55:552
1551599392,28cyclictest7950irq/56-dwc_otg_17:20:552
15515993911,5cyclictest26335-21diskmemload20:24:342
15515993911,5cyclictest26335-21diskmemload20:24:342
1551499397,19cyclictest221ktimersoftd/120:25:571
1551499394,5cyclictest21-21rcuc/118:53:031
1551499393,6cyclictest0-21swapper/118:00:531
1551499392,25cyclictest0-21swapper/119:09:551
1551499392,21cyclictest0-21swapper/120:41:021
1551499391,30cyclictest0-21swapper/116:43:481
1551499391,18cyclictest3965-21ssh20:35:561
74852380,2sleep10-21swapper/118:46:001
15515993820,11cyclictest7850irq/56-dwc_otg_20:17:162
15515993816,8cyclictest7950irq/56-dwc_otg_18:47:432
1551499388,7cyclictest21-21rcuc/118:41:591
1551499387,24cyclictest0-21swapper/118:05:551
1551499386,15cyclictest0-21swapper/121:00:581
15514993814,5cyclictest10152-21unixbench_multi19:26:091
15514993813,9cyclictest221ktimersoftd/120:54:091
1551499381,20cyclictest221ktimersoftd/120:45:561
1551699379,16cyclictest0-21swapper/316:40:533
1551699378,13cyclictest7750irq/56-dwc_otg21:15:423
1551699374,7cyclictest7850irq/56-dwc_otg_17:35:563
1551699371,30cyclictest3081-21ls19:56:083
1551699371,30cyclictest3081-21ls19:56:083
1551599377,9cyclictest29-21rcuc/220:37:142
1551599377,4cyclictest7850irq/56-dwc_otg_19:55:422
1551599376,19cyclictest0-21swapper/217:15:562
1551599374,6cyclictest29-21rcuc/221:48:202
1551599372,17cyclictest0-21swapper/219:10:552
1551599372,17cyclictest0-21swapper/219:10:552
15515993717,9cyclictest30449-21ssh20:28:332
15515993710,10cyclictest301ktimersoftd/221:45:072
1551499375,5cyclictest21-21rcuc/121:50:451
1551499374,6cyclictest21-21rcuc/121:50:411
1551499371,8cyclictest0-21swapper/121:18:301
1551499371,23cyclictest28581-21ls17:06:001
1551699368,13cyclictest7850irq/56-dwc_otg_19:51:103
1551699364,24cyclictest7950irq/56-dwc_otg_20:55:523
1551699363,12cyclictest0-21swapper/316:30:443
1551699361,6cyclictest12651-21df20:45:573
1551599368,7cyclictest7950irq/56-dwc_otg_21:51:092
1551599368,10cyclictest301ktimersoftd/219:57:362
1551599368,10cyclictest301ktimersoftd/219:57:362
1551599367,23cyclictest0-21swapper/221:55:562
1551599367,23cyclictest0-21swapper/221:55:562
15515993620,5cyclictest20164-21sh19:00:582
15515993611,19cyclictest0-21swapper/221:17:112
15515993611,11cyclictest7850irq/56-dwc_otg_18:36:362
1551499367,14cyclictest0-21swapper/116:50:551
1551499365,12cyclictest23-21ksoftirqd/118:12:271
1551499365,12cyclictest23-21ksoftirqd/118:12:271
1551499364,13cyclictest23-21ksoftirqd/120:31:231
15514993619,4cyclictest30241-21sshd17:10:591
1551499361,19cyclictest0-21swapper/118:00:211
1551499361,15cyclictest0-21swapper/121:36:031
1551499361,11cyclictest0-21swapper/119:55:551
1551499361,11cyclictest0-21swapper/119:55:551
15514993610,13cyclictest221ktimersoftd/120:57:181
1551699355,6cyclictest7750irq/56-dwc_otg20:11:103
1551699352,4cyclictest0-21swapper/320:09:093
1551699351,23cyclictest11081-21timerwakeupswit21:21:083
1551599359,5cyclictest13932-21rm20:09:332
1551599358,5cyclictest7850irq/56-dwc_otg_18:51:042
1551599357,18cyclictest0-21swapper/221:07:332
1551599355,7cyclictest9450irq/48-DMA09:12:252
1551599352,6cyclictest5122-21tune2fs17:30:592
1551599351,8cyclictest31-21ksoftirqd/219:35:002
1551599351,8cyclictest31-21ksoftirqd/219:35:002
15515993516,5cyclictest908-21runrttasks16:57:272
15515993516,5cyclictest908-21runrttasks16:57:272
1551599351,24cyclictest0-21swapper/218:00:552
1551499352,19cyclictest0-21swapper/119:30:501
1551499352,19cyclictest0-21swapper/119:30:501
1551699344,11cyclictest7850irq/56-dwc_otg_20:32:013
1551599349,5cyclictest10389-21ssh20:42:462
1551599347,8cyclictest301ktimersoftd/221:11:072
15515993410,8cyclictest7850irq/56-dwc_otg_17:36:062
1551499347,21cyclictest0-21swapper/117:25:561
1551499346,8cyclictest21-21rcuc/118:26:081
15514993412,5cyclictest22216-21grep18:16:091
1551699338,12cyclictest7950irq/56-dwc_otg_20:21:053
1551699338,12cyclictest7950irq/56-dwc_otg_20:21:053
1551699336,14cyclictest39-21ksoftirqd/317:21:093
1551699334,13cyclictest0-21swapper/319:21:433
1551699331,4cyclictest0-21swapper/321:36:003
15516993312,12cyclictest19960-21/usr/sbin/munin20:16:033
1551599333,14cyclictest7850irq/56-dwc_otg_16:40:582
15514993312,9cyclictest17892-21date16:36:021
1551699326,4cyclictest0-21swapper/321:07:343
1551699326,12cyclictest7750irq/56-dwc_otg19:00:593
1551699325,4cyclictest7750irq/56-dwc_otg19:35:533
1551699323,17cyclictest0-21swapper/321:34:003
15516993226,4cyclictest30536-21ls17:11:033
1551699322,5cyclictest0-21swapper/316:56:073
1551699322,5cyclictest0-21swapper/316:56:073
1551699322,17cyclictest7750irq/56-dwc_otg21:59:563
1551699322,17cyclictest7750irq/56-dwc_otg21:59:563
1551699321,5cyclictest6100-21irqstats21:16:013
1551699321,21cyclictest4536-21missed_timers20:36:033
1551599325,6cyclictest7950irq/56-dwc_otg_18:30:412
15515993216,5cyclictest19453-21aten_rbpower_vo18:10:572
15515993216,5cyclictest19453-21aten_rbpower_vo18:10:572
1551699319,3cyclictest0-21swapper/318:48:163
1551699316,12cyclictest7850irq/56-dwc_otg_18:11:073
1551699316,12cyclictest7850irq/56-dwc_otg_18:11:073
1551699311,5cyclictest7750irq/56-dwc_otg17:30:413
15516993114,10cyclictest381ktimersoftd/319:14:403
15516993114,10cyclictest381ktimersoftd/319:14:403
1551699311,17cyclictest0-21swapper/321:46:033
1551599313,5cyclictest9829-21ps19:26:062
1551699307,4cyclictest0-21swapper/319:44:143
1551699307,4cyclictest0-21swapper/319:10:223
1551699305,7cyclictest7750irq/56-dwc_otg21:27:013
1551699303,11cyclictest7950irq/56-dwc_otg_17:31:113
15516993015,6cyclictest17551-21/usr/sbin/munin18:06:073
1551599304,5cyclictest21155-21/usr/sbin/munin18:16:002
15515993014,4cyclictest30482-21cat17:11:022
1551699296,14cyclictest0-21swapper/317:45:483
1551699296,12cyclictest7850irq/56-dwc_otg_21:50:553
1551699293,12cyclictest7950irq/56-dwc_otg_16:46:083
1551699291,3cyclictest0-21swapper/320:00:583
1551699291,19cyclictest0-21swapper/316:36:143
1551699291,13cyclictest7950irq/56-dwc_otg_21:01:063
1551599293,9cyclictest7950irq/56-dwc_otg_17:06:102
1551699286,9cyclictest7950irq/56-dwc_otg_17:01:043
1551699285,5cyclictest7950irq/56-dwc_otg_18:01:093
15516992819,6cyclictest381ktimersoftd/321:41:003
1551699281,21cyclictest28746-21latency17:06:023
1551699281,21cyclictest21519-21idleruntime18:15:593
1551699275,4cyclictest7750irq/56-dwc_otg17:50:583
1551699273,12cyclictest7950irq/56-dwc_otg_16:50:563
1551699266,4cyclictest7850irq/56-dwc_otg_18:20:573
1551599261,7cyclictest7850irq/56-dwc_otg_16:36:042
150152264,15sleep10-21swapper/116:27:521
149462263,14sleep20-21swapper/216:27:002
1551699252,12cyclictest15103-21grep17:56:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional