You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 04:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Apr 26, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1704025010,5sleep10-21swapper/104:26:061
1704025010,5sleep10-21swapper/104:26:061
1699824200,5sleep00-21swapper/004:26:050
1699824200,5sleep00-21swapper/004:26:050
3098521760,5sleep21860299cyclictest07:48:302
107772880,4sleep20-21swapper/209:21:192
77112860,2sleep20-21swapper/206:42:072
6852850,2sleep20-21swapper/208:31:022
305332820,2sleep20-21swapper/207:09:412
308392780,2sleep30-21swapper/309:06:183
269922780,2sleep20-21swapper/207:43:432
269922780,2sleep20-21swapper/207:43:432
218902770,2sleep20-21swapper/206:59:262
1860099721,60cyclictest0-21swapper/009:11:060
317242700,2sleep30-21swapper/305:06:083
189522700,4sleep00-21swapper/004:31:070
1860099685,33cyclictest0-21swapper/008:51:020
1860099685,33cyclictest0-21swapper/008:51:020
18600996813,9cyclictest0-21swapper/007:11:020
18600996752,7cyclictest0-21swapper/008:46:030
18602996627,17cyclictest1982-21ssh09:50:562
1860099662,48cyclictest9-21ksoftirqd/007:15:590
1860099661,29cyclictest0-21swapper/009:06:030
255302650,7sleep11860199cyclictest09:01:011
18602996523,22cyclictest18200-21ssh07:32:462
18602996523,12cyclictest19730-21grep04:31:172
18602996514,8cyclictest7950irq/56-dwc_otg_08:26:022
18601996513,11cyclictest7950irq/56-dwc_otg_09:56:021
1860099657,42cyclictest0-21swapper/005:56:010
18600996521,16cyclictest0-21swapper/009:36:000
1860099642,47cyclictest13-21rcuc/005:00:500
1860099642,41cyclictest25923-21grep09:01:070
18600996422,20cyclictest0-21swapper/006:41:000
1860099633,16cyclictest7750irq/56-dwc_otg05:36:020
18600996246,6cyclictest7750irq/56-dwc_otg08:36:180
18600996236,7cyclictest7850irq/56-dwc_otg_05:06:180
1860099622,44cyclictest9662-21grep09:21:040
18600996213,5cyclictest0-21swapper/006:56:020
18600996143,7cyclictest7850irq/56-dwc_otg_09:50:500
1860099613,45cyclictest13725-21taskset09:26:020
18600996132,5cyclictest7950irq/56-dwc_otg_07:26:030
18602996025,17cyclictest12488-21ssh08:05:592
18602996025,10cyclictest29577-21apt-get05:01:002
18601996028,15cyclictest1980-21apt-get09:50:561
1860099602,44cyclictest20459-21awk08:15:490
18600996015,27cyclictest0-21swapper/005:11:010
18601995935,8cyclictest7958-21apt-get05:31:001
18601995917,22cyclictest1639-21ssh07:13:261
1860099598,24cyclictest0-21swapper/008:01:030
18602995823,13cyclictest22839-21ssh07:38:402
18602995819,12cyclictest917-21snmpd06:51:102
18601995820,16cyclictest29611-21aten_rbpower_en05:01:031
18600995817,10cyclictest0-21swapper/007:36:000
1860099581,50cyclictest0-21swapper/008:10:490
1860099581,36cyclictest7778-21latency_hist05:30:500
18600995812,4cyclictest0-21swapper/006:51:030
18602995728,11cyclictest2188-21sshd05:14:322
18602995726,15cyclictest18316-21rm08:11:562
18602995725,20cyclictest24167-21ssh08:58:562
18601995718,24cyclictest31630-21ssh07:49:411
18600995737,12cyclictest0-21swapper/009:32:390
18600995726,15cyclictest24736-21grep04:46:120
141272570,3sleep121-21rcuc/107:27:521
18602995624,20cyclictest27064-21ssh09:01:422
18602995619,23cyclictest17273-21ssh09:30:252
18602995618,23cyclictest26975-21ssh08:22:432
1860199566,12cyclictest7950irq/56-dwc_otg_06:31:021
18601995616,24cyclictest7750irq/56-dwc_otg06:51:131
18601995615,25cyclictest28441-21latency06:26:091
18601995615,21cyclictest24235-21ssh08:20:151
18601995615,12cyclictest9830-21/usr/sbin/munin09:21:061
18601995614,11cyclictest18616-21/usr/sbin/munin04:31:171
18600995632,13cyclictest10222-21unixbench-2d08:41:170
1860099562,38cyclictest0-21swapper/004:51:050
1860099562,37cyclictest3350-21unixbench_singl09:51:210
18600995615,6cyclictest0-21swapper/007:01:010
18600995615,6cyclictest0-21swapper/007:01:010
1860099561,48cyclictest0-21swapper/006:39:500
1860099561,46cyclictest0-21swapper/009:56:190
1860099561,44cyclictest0-21swapper/009:16:050
1860099561,44cyclictest0-21swapper/007:57:320
18600995610,25cyclictest0-21swapper/005:41:130
18600995610,25cyclictest0-21swapper/005:41:130
234812550,2sleep10-21swapper/107:39:511
18602995530,16cyclictest4132-21ntp_states07:16:122
18602995518,24cyclictest7950irq/56-dwc_otg_09:40:242
18601995515,24cyclictest3279-21ssh09:12:271
1860099552,38cyclictest0-21swapper/005:46:110
1860099551,40cyclictest22154-21ntp_kernel_err08:56:110
18602995423,16cyclictest6312-21ssh08:36:292
18602995419,23cyclictest9351-21rm07:22:012
18602995419,23cyclictest9351-21rm07:22:012
1860199548,10cyclictest7950irq/56-dwc_otg_08:31:021
18601995421,23cyclictest18320-21ssh08:11:571
18601995414,25cyclictest26735-21ssh07:05:281
18601995414,25cyclictest26735-21ssh07:05:281
1860299533,9cyclictest7850irq/56-dwc_otg_09:56:022
18602995326,19cyclictest3630-21ssh07:55:172
18602995326,13cyclictest31273-21ssh09:06:512
18602995321,14cyclictest26450-21ssh07:04:592
18602995321,14cyclictest26450-21ssh07:04:592
18602995320,18cyclictest10912-21ssh08:42:182
18602995319,11cyclictest20874-21taskset09:34:252
18602995318,10cyclictest7850irq/56-dwc_otg_05:56:072
18602995317,27cyclictest4954-21ssh09:15:322
18601995316,20cyclictest10790-21ssh08:03:181
1860099532,26cyclictest26184-21grep08:21:200
1860099532,12cyclictest0-21swapper/005:51:000
18600995318,20cyclictest22287-21ls06:10:490
18602995224,16cyclictest31376-21ssh06:31:392
18602995222,20cyclictest10163-21acpi06:46:002
18602995222,20cyclictest10163-21acpi06:46:002
18602995222,19cyclictest5336-21ssh06:40:242
18602995221,12cyclictest19368-21irqstats06:01:082
18602995220,13cyclictest8159-21runrttasks05:31:042
18602995218,18cyclictest14032-21ssh07:27:412
18602995216,11cyclictest31539-21apt07:11:012
18602995215,11cyclictest15392-21ssh08:47:572
1860199525,5cyclictest7850irq/56-dwc_otg_08:26:021
1860199524,9cyclictest7950irq/56-dwc_otg_08:36:021
18601995216,20cyclictest7893-21ssh08:00:321
18601995214,12cyclictest28663-21cron08:25:491
18601995212,18cyclictest2100-21ssh07:52:271
18601995211,10cyclictest7950irq/56-dwc_otg_09:37:201
1860099524,32cyclictest30283-21unixbench_singl08:26:190
18600995240,4cyclictest7850irq/56-dwc_otg_07:25:490
18600995240,4cyclictest7850irq/56-dwc_otg_07:25:490
18600995227,6cyclictest7850irq/56-dwc_otg_05:00:520
18602995127,14cyclictest5870-21sshd05:25:322
18602995122,13cyclictest29704-21ssh09:45:282
18602995118,12cyclictest10999-21ssh08:03:402
18602995117,23cyclictest7858-21ssh09:18:192
18602995117,14cyclictest24003-21ssh08:19:492
18602995113,22cyclictest27390-21chrt04:55:402
18601995120,20cyclictest18594-21ssh07:33:291
18601995117,21cyclictest20384-21ssh06:56:461
18601995117,20cyclictest6178-21ssh07:18:561
18601995117,17cyclictest12518-21dpkg08:06:001
18601995115,22cyclictest22044-21memory08:56:091
18601995112,16cyclictest10082-21ssh06:45:521
18601995112,16cyclictest10082-21ssh06:45:521
18600995136,6cyclictest7750irq/56-dwc_otg08:20:490
1860099512,28cyclictest0-21swapper/005:16:020
1860099511,5cyclictest632-21/usr/sbin/munin08:31:180
18602995024,18cyclictest27464-21sshd06:23:172
18602995023,11cyclictest6773-21ssh07:58:292
18602995018,18cyclictest0-21swapper/206:28:552
18601995019,17cyclictest323-21ssh09:48:331
18601995019,15cyclictest30124-21date09:45:501
18601995017,11cyclictest7669-21sh09:17:581
18600995030,11cyclictest0-21swapper/006:48:120
18600995030,11cyclictest0-21swapper/006:48:120
18602994921,16cyclictest10963-21taskset05:36:312
18602994919,10cyclictest7950irq/56-dwc_otg_06:06:462
18602994918,11cyclictest6138-21/usr/sbin/munin05:26:152
18602994916,17cyclictest7850irq/56-dwc_otg_09:48:192
18602994915,22cyclictest7950irq/56-dwc_otg_04:45:102
18602994913,26cyclictest0-21swapper/204:36:412
1860199493,11cyclictest7750irq/56-dwc_otg04:41:021
18601994918,21cyclictest9174-21ssh07:21:421
18601994918,21cyclictest9174-21ssh07:21:421
18601994918,17cyclictest19386-21ssh09:31:421
18601994917,20cyclictest27229-21ssh07:44:101
18601994917,20cyclictest27229-21ssh07:44:101
18601994914,20cyclictest14741-21expr09:26:171
18601994911,17cyclictest5907-21sh05:25:491
18601994911,10cyclictest1989-21fschecks_time06:36:071
18600994919,5cyclictest22016-21munin-plugin-st04:40:490
18600994916,20cyclictest32576-21/usr/sbin/munin07:51:160
1860099491,37cyclictest0-21swapper/007:09:190
18600994912,22cyclictest13-21rcuc/006:12:170
1860099481,23cyclictest20465-21latency_hist06:05:500
1860399472,10cyclictest0-21swapper/304:41:003
18602994718,17cyclictest352-21sshd05:09:322
18602994715,12cyclictest3837-21taskset05:17:382
18602994714,21cyclictest17825-21ssh08:51:102
18602994714,21cyclictest17825-21ssh08:51:102
18601994721,16cyclictest0-21swapper/107:08:111
18601994719,12cyclictest16863-21rm08:50:401
18601994713,20cyclictest27542-1kworker/u9:209:06:361
18601994712,10cyclictest7750irq/56-dwc_otg06:42:521
1860099471,14cyclictest6009-21latency_hist05:25:500
1860399467,25cyclictest0-21swapper/305:31:033
18601994618,15cyclictest10760-21ssh08:42:021
18601994615,18cyclictest21030-21rm08:55:441
18601994615,18cyclictest21030-21rm08:55:441
18601994612,8cyclictest17427-21perl05:56:061
1860099463,16cyclictest0-21swapper/007:41:030
1860099463,16cyclictest0-21swapper/007:41:030
1860099461,35cyclictest0-21swapper/007:35:240
1860399457,23cyclictest0-21swapper/305:41:033
1860399457,23cyclictest0-21swapper/305:41:033
18602994511,10cyclictest7950irq/56-dwc_otg_05:55:502
1860199456,29cyclictest7950irq/56-dwc_otg_04:36:011
18601994517,17cyclictest13068-21runrttasks05:45:051
18601994517,17cyclictest13068-21runrttasks05:45:051
18601994512,11cyclictest7750irq/56-dwc_otg05:55:491
18601994511,14cyclictest19370-21/usr/sbin/munin06:01:081
1860099456,25cyclictest0-21swapper/006:16:020
1860099456,25cyclictest0-21swapper/006:16:020
1860099456,23cyclictest0-21swapper/004:41:020
1860099452,31cyclictest0-21swapper/009:45:090
1860099452,24cyclictest908-21runrttasks07:48:400
1860099451,32cyclictest0-21swapper/006:33:420
1860399442,8cyclictest7950irq/56-dwc_otg_05:56:003
18602994411,12cyclictest7850irq/56-dwc_otg_06:18:102
18602994411,12cyclictest7850irq/56-dwc_otg_06:18:102
18601994416,12cyclictest321-21chrt05:09:071
1860399437,24cyclictest0-21swapper/305:00:553
1860399437,22cyclictest0-21swapper/306:06:043
18602994317,10cyclictest0-21swapper/205:42:202
18602994317,10cyclictest0-21swapper/205:42:202
18602994315,13cyclictest25358-21seq04:47:452
18601994312,18cyclictest7814-21latency_hist05:30:501
18601994312,13cyclictest0-21swapper/104:47:311
18600994319,11cyclictest0-21swapper/005:33:560
1860399422,5cyclictest7850irq/56-dwc_otg_08:30:493
18602994226,13cyclictest22778-21fschecks_count06:11:062
1860199428,15cyclictest0-21swapper/105:12:181
1860199427,7cyclictest7850irq/56-dwc_otg_05:36:001
18601994218,11cyclictest4063-21taskset05:20:251
18601994214,11cyclictest14658-21taskset05:47:251
18601994212,14cyclictest7950irq/56-dwc_otg_04:53:181
18602994115,11cyclictest7950irq/56-dwc_otg_04:56:012
1860199416,23cyclictest0-21swapper/106:11:031
1860399407,24cyclictest0-21swapper/309:46:023
1860199409,7cyclictest7950irq/56-dwc_otg_04:56:011
18601994012,15cyclictest25939-21cut06:20:491
18601994012,15cyclictest25939-21cut06:20:491
1860099402,26cyclictest0-21swapper/006:26:280
219382391,33sleep30-21swapper/307:37:053
1860399392,7cyclictest0-21swapper/309:21:013
1860399386,23cyclictest0-21swapper/307:25:563
18602993824,10cyclictest14589-21sshd05:46:322
18600993815,11cyclictest0-21swapper/006:20:540
1860399377,6cyclictest7950irq/56-dwc_otg_08:36:013
1860399368,19cyclictest0-21swapper/308:41:033
1860399368,13cyclictest0-21swapper/309:00:493
1860399361,5cyclictest7950irq/56-dwc_otg_08:46:013
18603993614,6cyclictest6883-21proc_pri05:26:153
1860399360,23cyclictest7950irq/56-dwc_otg_09:41:083
18601993613,11cyclictest27496-21taskset06:23:391
1860399357,15cyclictest0-21swapper/307:21:043
1860399357,15cyclictest0-21swapper/307:21:043
1860399356,16cyclictest39-21ksoftirqd/307:56:123
1860399355,7cyclictest7950irq/56-dwc_otg_08:16:183
1860399353,13cyclictest7950irq/56-dwc_otg_06:25:493
18603993520,4cyclictest18376-21kernelversion09:31:103
1860399351,25cyclictest4497-21ls08:35:493
1860399351,23cyclictest19196-21ls06:56:073
1860399343,5cyclictest0-21swapper/309:51:143
1860399342,19cyclictest0-21swapper/307:43:553
1860399342,19cyclictest0-21swapper/307:43:553
1860399341,22cyclictest19825-21timerandwakeup06:01:153
1860399341,13cyclictest0-21swapper/307:34:053
18603993410,7cyclictest18505-21unixbench-2d08:51:193
18603993410,7cyclictest18505-21unixbench-2d08:51:193
1860399339,4cyclictest0-21swapper/309:20:503
1860399339,16cyclictest13104-21latency08:06:103
1860399334,20cyclictest32597-21ssh07:51:013
1860399334,13cyclictest7950irq/56-dwc_otg_04:31:163
18603993312,6cyclictest4855-1kworker/u9:207:20:493
18601993319,11cyclictest21339-21sed06:06:111
1860399327,5cyclictest7850irq/56-dwc_otg_06:40:503
1860399327,15cyclictest0-21swapper/309:01:123
1860399326,13cyclictest7850irq/56-dwc_otg_06:31:203
1860399324,14cyclictest7950irq/56-dwc_otg_05:11:033
1860399322,20cyclictest0-21swapper/309:58:453
18603993216,4cyclictest7850irq/56-dwc_otg_07:46:053
1860399321,26cyclictest23431-21ls07:01:073
1860399321,26cyclictest23431-21ls07:01:073
1860399321,22cyclictest9850-21latency_hist06:45:503
1860399321,21cyclictest10750-21ls05:36:163
18603993210,4cyclictest0-21swapper/307:07:343
1860399318,12cyclictest7850irq/56-dwc_otg_05:16:163
1860399317,12cyclictest7750irq/56-dwc_otg06:51:103
1860399313,5cyclictest7750irq/56-dwc_otg06:46:123
1860399313,5cyclictest7750irq/56-dwc_otg06:46:123
1860399312,18cyclictest0-21swapper/308:10:593
1860399311,6cyclictest14475-21unixbench_multi05:46:173
18603993111,10cyclictest32683-21/usr/sbin/munin07:11:183
1860399304,5cyclictest7850irq/56-dwc_otg_08:01:173
1860399303,5cyclictest7750irq/56-dwc_otg05:00:493
1860399302,4cyclictest1319-21apt-get09:11:013
1860399302,19cyclictest0-21swapper/308:22:193
1860399301,23cyclictest28912-21timerwakeupswit06:26:173
1860399301,20cyclictest14679-21ssh09:26:173
1860399301,14cyclictest0-21swapper/306:15:503
18603993010,12cyclictest0-21swapper/309:37:483
1860399293,4cyclictest0-21swapper/304:55:503
1860399292,14cyclictest7950irq/56-dwc_otg_04:36:083
1860399284,14cyclictest7950irq/56-dwc_otg_06:16:063
1860399284,14cyclictest7950irq/56-dwc_otg_06:16:063
182962284,14sleep20-21swapper/204:30:252
182962284,14sleep20-21swapper/204:30:252
1860399276,6cyclictest7850irq/56-dwc_otg_05:55:493
1860399271,21cyclictest25039-21timerwakeupswit04:46:173
169822270,2sleep30-21swapper/304:26:043
169822270,2sleep30-21swapper/304:26:043
43422251,4sleep30-21swapper/305:21:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional