You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-13 - 03:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Mon Apr 13, 2026 00:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1319222980,2sleep00-21swapper/004:26:080
1319222980,2sleep00-21swapper/004:26:080
304021600,4sleep11427699cyclictest07:57:291
234052950,3sleep30-21swapper/304:55:563
39812860,2sleep20-21swapper/207:59:072
125062810,3sleep012505-21ssh06:52:580
286672800,2sleep00-21swapper/009:06:090
14275997835,15cyclictest7950irq/56-dwc_otg_09:35:530
1427599756,60cyclictest0-21swapper/007:32:260
287932720,2sleep20-21swapper/206:34:482
287932720,2sleep20-21swapper/206:34:482
14275997243,12cyclictest23983-21/usr/sbin/munin07:06:090
67512710,2sleep30-21swapper/309:19:453
1427599711,57cyclictest0-21swapper/006:10:420
205882700,2sleep20-21swapper/207:40:512
14275997010,49cyclictest27703-21fschecks_time07:11:010
203412690,2sleep00-21swapper/004:46:020
14275996950,11cyclictest101ktimersoftd/009:41:020
1427599692,52cyclictest0-21swapper/007:16:100
14275996912,44cyclictest7345-21idleruntime-cro07:25:420
14275996912,41cyclictest0-21swapper/005:11:120
153092670,9sleep1221ktimersoftd/107:34:301
1427799673,19cyclictest7750irq/56-dwc_otg08:20:542
1427599671,49cyclictest28614-21runrttasks05:10:420
1427699664,37cyclictest0-21swapper/105:00:481
1427599661,52cyclictest0-21swapper/005:46:030
1427599661,52cyclictest0-21swapper/005:46:030
1427599656,36cyclictest0-21swapper/006:16:110
1427599651,56cyclictest1820-21kworker/u8:306:56:110
1427599651,56cyclictest1820-21kworker/u8:306:56:110
14275996344,6cyclictest101ktimersoftd/009:15:420
1427599634,43cyclictest9-21ksoftirqd/007:50:430
1427599634,43cyclictest9-21ksoftirqd/007:50:430
1427599631,10cyclictest3380-21latency_hist05:30:420
1427599624,47cyclictest22474-21ssh08:20:570
14275996242,13cyclictest101ktimersoftd/004:40:420
1427599622,47cyclictest0-21swapper/004:41:070
14275996123,31cyclictest0-21swapper/006:40:470
22892600,6sleep21427799cyclictest07:19:212
14275996027,6cyclictest101ktimersoftd/005:20:570
1427599602,47cyclictest0-21swapper/004:30:580
14277995924,7cyclictest24125-21ls07:06:102
1427699592,34cyclictest7850irq/56-dwc_otg_07:40:551
1427599595,7cyclictest14214-21kworker/u8:206:30:420
1427599594,43cyclictest0-21swapper/008:45:420
14275995939,12cyclictest101ktimersoftd/009:52:030
14275995919,27cyclictest5219-21/usr/sbin/munin08:01:050
14275995912,36cyclictest11439-21modprobe09:25:430
14277995821,19cyclictest20537-21/usr/sbin/munin06:16:042
14277995818,14cyclictest30418-21unixbench_singl07:51:112
14277995813,29cyclictest25054-21diskmemload06:40:562
14275995832,12cyclictest11620-21grep05:51:080
1427599583,19cyclictest0-21swapper/008:05:540
1427599581,33cyclictest28493-21ssh09:45:570
1427799574,14cyclictest11857-21ssh09:25:542
14275995740,7cyclictest19645-21idleruntime-cro06:15:420
14275995732,8cyclictest29899-21/usr/sbin/munin07:51:050
14277995620,11cyclictest7434-21ssh08:02:422
14277995616,21cyclictest7750irq/56-dwc_otg09:41:052
1427599564,37cyclictest13-21rcuc/008:36:090
14275995634,4cyclictest101ktimersoftd/008:56:010
14275995623,4cyclictest0-21swapper/008:31:030
14277995522,21cyclictest4141-21rm08:36:362
14277995518,12cyclictest29549-21ssh09:07:162
1427699555,29cyclictest29171-21ssh07:50:561
14275995540,8cyclictest0-21swapper/009:25:490
1427599552,34cyclictest23248-21sh06:25:420
82542540,3sleep38258-21latency_hist10:00:433
82542540,3sleep38258-21latency_hist10:00:433
14275995440,3cyclictest0-21swapper/010:00:420
14275995440,3cyclictest0-21swapper/010:00:420
1427599543,38cyclictest0-21swapper/008:50:490
14275995433,5cyclictest7750irq/56-dwc_otg09:20:430
14275995432,11cyclictest101ktimersoftd/007:25:490
1427599541,25cyclictest1704-21awk07:56:040
1427599541,24cyclictest26984-21tune2fs08:26:010
1427599541,15cyclictest0-21swapper/006:40:430
14277995324,13cyclictest32673-21diskstats05:20:592
14277995318,11cyclictest20011-21date09:35:442
1427599535,38cyclictest0-21swapper/007:35:530
14277995218,14cyclictest10480-21taskset06:51:002
14277995218,13cyclictest30740-21latency_hist08:30:432
14277995215,12cyclictest7693-21ssh07:25:482
1427599526,34cyclictest9-21ksoftirqd/008:10:460
1427599521,8cyclictest6535-21unixbench_multi05:36:100
1427599521,8cyclictest25473-21fschecks_count05:00:580
1427599521,40cyclictest0-21swapper/009:00:580
1427799516,10cyclictest7850irq/56-dwc_otg_05:30:552
1427799513,8cyclictest0-21swapper/206:20:532
14277995123,15cyclictest13878-21apt-get08:10:552
14277995118,15cyclictest31478-21ssh09:49:212
14277995116,25cyclictest7850irq/56-dwc_otg_08:19:472
14277995115,23cyclictest7750irq/56-dwc_otg08:42:002
1427599511,8cyclictest29432-21grep06:35:430
1427599511,8cyclictest29432-21grep06:35:430
1427599511,30cyclictest24212-21grep04:56:080
14277995034,12cyclictest27053-21apt-get05:05:552
14277995020,16cyclictest28235-21ssh07:49:432
14277995020,16cyclictest28235-21ssh07:49:432
14277995020,13cyclictest19736-21cut06:15:432
14277995019,19cyclictest8978-21sh06:49:162
14277995019,13cyclictest24996-21taskset09:01:422
14277995017,22cyclictest9240-21ssh09:21:442
14277995017,13cyclictest31542-21memory08:31:032
14277995015,23cyclictest7750irq/56-dwc_otg08:53:142
1427599503,26cyclictest0-21swapper/005:41:050
14275995027,5cyclictest7750irq/56-dwc_otg08:15:530
1427599502,6cyclictest12652-21acpi05:55:540
14275995026,10cyclictest0-21swapper/005:15:590
1427599502,28cyclictest13-21rcuc/004:50:590
14275995017,23cyclictest18705-21sh07:00:550
14275995015,22cyclictest13-21rcuc/008:46:340
1427799499,15cyclictest9401-21/usr/sbin/munin05:46:002
1427799499,15cyclictest9401-21/usr/sbin/munin05:46:002
14277994920,13cyclictest11339-21kernelversion05:51:042
14277994917,17cyclictest18769-21cut07:00:562
14277994911,27cyclictest3524-21sh09:54:592
1427699491,20cyclictest0-21swapper/106:36:041
1427599497,25cyclictest9-21ksoftirqd/007:45:230
14275994926,4cyclictest0-21swapper/006:00:530
1427799481,9cyclictest0-21swapper/208:45:532
14277994817,20cyclictest4182-21ssh09:55:452
14277994817,20cyclictest4182-21ssh09:55:452
14276994811,25cyclictest0-21swapper/105:45:561
14276994811,25cyclictest0-21swapper/105:45:561
14278994720,6cyclictest16021-21cron06:05:423
1427799477,26cyclictest0-21swapper/209:15:562
1427799477,25cyclictest0-21swapper/204:40:552
14277994717,13cyclictest15175-21ssh07:34:162
14277994715,16cyclictest30420-21basename05:15:422
14277994715,13cyclictest20020-21ntp_kernel_pll_08:56:052
14277994713,12cyclictest29022-21ssh07:11:502
1427599472,36cyclictest0-21swapper/006:46:200
14277994616,13cyclictest7850irq/56-dwc_otg_05:45:432
14277994616,12cyclictest23026-21ssh09:39:022
14277994612,25cyclictest7850irq/56-dwc_otg_05:16:042
14277994611,11cyclictest0-21swapper/208:08:062
1427699461,35cyclictest0-21swapper/109:53:061
14275994619,7cyclictest101ktimersoftd/009:35:420
14278994518,6cyclictest37-21rcuc/308:10:423
1427799456,12cyclictest29684-21ssh06:35:562
14277994517,14cyclictest20469-21ps04:46:062
1427799449,25cyclictest6017-21ssh07:23:252
1427799447,24cyclictest0-21swapper/207:35:552
1427799447,24cyclictest0-21swapper/206:05:562
14277994415,18cyclictest14108-21chrt05:59:222
14277994415,11cyclictest17433-21ssh06:59:062
14277994415,11cyclictest17433-21ssh06:59:062
14277994413,20cyclictest15638-21taskset04:33:012
1427699441,26cyclictest0-21swapper/105:10:571
1427699438,23cyclictest7750irq/56-dwc_otg07:25:421
1427699437,25cyclictest0-21swapper/108:45:551
1427699436,24cyclictest24368-21ls06:26:081
1427699431,28cyclictest0-21swapper/108:40:561
1427699431,16cyclictest7950irq/56-dwc_otg_04:51:071
1427899423,5cyclictest7950irq/56-dwc_otg_08:50:543
1427799427,15cyclictest7850irq/56-dwc_otg_05:40:432
14277994218,13cyclictest26716-21taskset05:04:512
14277994214,11cyclictest1608-21ssh09:12:562
1427699421,13cyclictest7850irq/56-dwc_otg_07:06:001
14275994214,13cyclictest0-21swapper/005:35:390
14277994110,11cyclictest0-21swapper/206:26:592
1427699411,5cyclictest0-21swapper/104:45:541
1427699411,23cyclictest0-21swapper/108:25:551
1427899408,22cyclictest0-21swapper/307:00:553
1427899401,5cyclictest7950irq/56-dwc_otg_04:50:533
14277994024,12cyclictest24242-21ls04:56:082
14277994014,17cyclictest15757-21runrttasks06:02:262
1427699406,4cyclictest10150-21apt-get06:50:551
1427699401,7cyclictest0-21swapper/108:10:481
14277993911,9cyclictest0-21swapper/205:26:272
1427699397,22cyclictest0-21swapper/108:50:491
1427699394,24cyclictest6946-21ls06:46:091
1427699392,23cyclictest7850irq/56-dwc_otg_08:01:061
14276993915,6cyclictest29653-21unixbench_multi09:46:111
14276993914,12cyclictest221ktimersoftd/109:18:551
1427699391,3cyclictest7750irq/56-dwc_otg07:15:551
1427699391,24cyclictest0-21swapper/105:25:551
1427699391,14cyclictest10694-21rm08:06:111
14277993823,12cyclictest21506-21/usr/sbin/munin04:51:072
1427699387,5cyclictest23-21ksoftirqd/107:00:431
1427699387,5cyclictest23-21ksoftirqd/107:00:431
1427699387,19cyclictest7750irq/56-dwc_otg07:26:031
1427699386,20cyclictest7750irq/56-dwc_otg09:21:121
1427699382,19cyclictest0-21swapper/105:50:581
1427699381,9cyclictest7850irq/56-dwc_otg_07:47:371
1427699381,9cyclictest7850irq/56-dwc_otg_07:47:371
1427699381,26cyclictest0-21swapper/104:55:551
1427699381,22cyclictest0-21swapper/105:40:561
14278993729,6cyclictest39-21ksoftirqd/307:36:083
14278993724,5cyclictest285872sleep305:10:423
14278993720,5cyclictest11387-21seq07:30:103
14277993714,8cyclictest17680-21/usr/sbin/munin04:40:422
1427699374,10cyclictest8096-21apt-key10:00:431
1427699374,10cyclictest8096-21apt-key10:00:431
1427699371,26cyclictest18990-21uniq07:00:591
1427699371,24cyclictest7750irq/56-dwc_otg08:56:121
1427699371,23cyclictest19811-21latency_hist09:35:431
137432370,5sleep30-21swapper/304:27:243
137432370,5sleep30-21swapper/304:27:243
1427899367,16cyclictest39-21ksoftirqd/308:11:083
1427899363,11cyclictest30692-21ssh09:09:203
14278993614,4cyclictest39-21ksoftirqd/308:55:543
14278993613,5cyclictest39-21ksoftirqd/307:45:433
1427699365,9cyclictest7950irq/56-dwc_otg_06:40:431
1427699363,5cyclictest7950irq/56-dwc_otg_08:20:551
1427699362,5cyclictest0-21swapper/108:31:031
1427699361,5cyclictest6582-21grep05:36:101
1427699361,25cyclictest0-21swapper/107:10:571
1427699361,19cyclictest20981-21grep06:16:101
1427899358,16cyclictest39-21ksoftirqd/309:41:073
14278993517,6cyclictest0-21swapper/307:55:383
14278993512,9cyclictest39-21ksoftirqd/304:40:563
14278993512,5cyclictest5696-21latency_hist06:45:433
1427699355,20cyclictest24471-21ls09:01:101
1427699354,20cyclictest23-21ksoftirqd/104:35:421
1427699354,19cyclictest21-21rcuc/104:41:011
14276993518,5cyclictest28013-21fschecks_time09:06:001
1427699351,14cyclictest0-21swapper/108:36:071
14276993511,13cyclictest0-21swapper/109:38:481
1427899349,6cyclictest37-21rcuc/307:09:103
14278993423,8cyclictest7950irq/56-dwc_otg_07:45:563
14278993423,8cyclictest7950irq/56-dwc_otg_07:45:563
14278993421,4cyclictest39-21ksoftirqd/309:54:493
14278993418,6cyclictest39-21ksoftirqd/307:00:433
14278993418,6cyclictest39-21ksoftirqd/307:00:433
14278993416,5cyclictest29208-21ssh07:12:093
14278993411,13cyclictest381ktimersoftd/308:31:033
1427699349,13cyclictest21-21rcuc/105:34:351
1427699345,22cyclictest7850irq/56-dwc_otg_06:21:001
1427699344,21cyclictest7850irq/56-dwc_otg_08:20:421
1427699341,17cyclictest0-21swapper/109:26:021
1427899339,5cyclictest23272-21latency_hist06:25:433
1427899334,6cyclictest7950irq/56-dwc_otg_05:51:013
14278993326,5cyclictest10119-21rm09:23:213
14278993321,8cyclictest28494-21aten_rbpower_en09:45:573
1427899331,6cyclictest15135-21ssh09:29:473
1427899331,5cyclictest7950irq/56-dwc_otg_09:30:563
14278993312,4cyclictest39-21ksoftirqd/306:32:473
14278993312,4cyclictest39-21ksoftirqd/306:32:473
1427699335,14cyclictest7750irq/56-dwc_otg05:06:011
1427699331,5cyclictest0-21swapper/109:41:391
1427699331,16cyclictest32447-21/usr/sbin/munin05:20:531
1427899328,11cyclictest0-21swapper/304:50:343
1427899326,14cyclictest39-21ksoftirqd/309:04:213
14278993225,5cyclictest39-21ksoftirqd/308:00:423
14278993220,10cyclictest908-21runrttasks08:42:263
14278993219,4cyclictest30639-21/usr/sbin/munin05:16:043
14278993214,9cyclictest530-21munin-node05:40:533
14278993213,10cyclictest908-21runrttasks08:01:063
1427899321,22cyclictest0-21swapper/306:39:223
14278993212,11cyclictest13895-21taskset05:56:483
1427899321,13cyclictest0-21swapper/308:27:303
14278993211,13cyclictest193192chrt06:11:473
1427699327,17cyclictest7850irq/56-dwc_otg_06:10:421
1427699322,16cyclictest0-21swapper/109:11:071
1427699321,22cyclictest7950irq/56-dwc_otg_07:35:501
1427699321,17cyclictest29328-21idleruntime-cro06:35:431
1427699321,17cyclictest29328-21idleruntime-cro06:35:431
1427899316,10cyclictest7850irq/56-dwc_otg_08:36:103
14278993124,5cyclictest39-21ksoftirqd/305:50:433
14278993124,5cyclictest39-21ksoftirqd/305:50:433
14278993114,5cyclictest7327-21latency_hist07:25:433
14278993113,6cyclictest39-21ksoftirqd/306:15:593
14278993110,13cyclictest0-21swapper/308:22:243
14278993110,11cyclictest39-21ksoftirqd/307:17:193
1427699311,22cyclictest7750irq/56-dwc_otg06:04:061
14278993024,4cyclictest39-21ksoftirqd/309:14:403
1427899301,4cyclictest12124-21cut07:30:563
1427899301,3cyclictest20132-21apt-get09:35:543
14278993012,6cyclictest22066-21munin-plugin-st08:20:423
1427899301,20cyclictest0-21swapper/306:55:053
1427699307,12cyclictest12773-21aten_rbpower_vo05:55:571
1427699301,10cyclictest19207-21sed06:11:121
1427899297,11cyclictest7950irq/56-dwc_otg_06:26:053
1427899292,10cyclictest7950irq/56-dwc_otg_04:31:043
1427899291,4cyclictest37-21rcuc/304:40:423
1427899291,24cyclictest5381-21ls05:35:433
14278992911,9cyclictest25054-21diskmemload06:49:053
1427899291,16cyclictest0-21swapper/308:45:553
1427699292,18cyclictest7750irq/56-dwc_otg05:16:031
1427699292,11cyclictest0-21swapper/104:39:161
1427899281,6cyclictest17848-21awk06:10:423
130272286,12sleep10-21swapper/104:26:061
130272286,12sleep10-21swapper/104:26:061
14278992720,5cyclictest7014-21munin-plugin-st05:40:423
1427899271,15cyclictest0-21swapper/305:01:013
1427899268,10cyclictest3519-21cat05:30:433
1427899263,4cyclictest7850irq/56-dwc_otg_05:20:573
14278992617,6cyclictest30414-21idleruntime-cro05:15:423
137972263,14sleep20-21swapper/204:28:052
137972263,14sleep20-21swapper/204:28:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional