You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-24 - 18:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Jul 24, 2026 12:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
161959915947,90cyclictest18086-21grep10:55:540
16195991552,127cyclictest27174-21diskmemload11:40:400
16195991431,79cyclictest0-21swapper/012:30:360
16195991431,79cyclictest0-21swapper/012:30:360
161959913043,81cyclictest646-21rt-features09:55:550
161959912927,71cyclictest0-21swapper/011:35:470
16196991235,87cyclictest19068-21cat12:15:521
16196991235,87cyclictest19068-21cat12:15:521
16196991143,81cyclictest32169-21apt-get09:55:411
16196991127,64cyclictest13967-21ntp_states11:30:481
161959911034,55cyclictest0-21swapper/012:15:500
161959911034,55cyclictest0-21swapper/012:15:500
16195991097,85cyclictest0-21swapper/009:30:360
16195991097,85cyclictest0-21swapper/009:30:360
161959910932,70cyclictest30401-21ssh11:50:480
161959910756,36cyclictest7950irq/56-dwc_otg_11:30:460
161959910735,65cyclictest28458-21latency_hist10:30:250
161959910735,65cyclictest28458-21latency_hist10:30:250
16195991065,70cyclictest0-21swapper/012:35:450
16197991032,9cyclictest7850irq/56-dwc_otg09:55:542
16196991023,70cyclictest27174-21diskmemload11:35:471
16195991006,60cyclictest0-21swapper/012:05:280
161959910027,57cyclictest28756-21grep09:10:510
161959910025,61cyclictest16876-21tail09:35:510
1619899998,13cyclictest8050irq/56-dwc_otg_11:40:383
16195999726,59cyclictest0-21swapper/008:10:370
16195999525,51cyclictest5038-21unixbench_multi10:00:510
16195999524,56cyclictest22219-21cut12:20:380
98162930,6sleep31619899cyclictest11:25:473
16196999312,56cyclictest0-21swapper/111:40:371
89512920,9sleep0101ktimersoftd/011:25:240
16198999112,18cyclictest0-21swapper/309:55:553
1619799912,13cyclictest19078-21ls12:15:522
1619799912,13cyclictest19078-21ls12:15:522
16195999126,48cyclictest17573-21ssh12:15:230
16195999122,53cyclictest27028-21rm12:25:490
16195999120,59cyclictest0-21swapper/011:49:380
16195999120,59cyclictest0-21swapper/011:49:380
1619699903,67cyclictest0-21swapper/112:35:431
1619699892,57cyclictest18006-21ssh10:55:531
16195998924,51cyclictest12379-21cron10:50:220
1619799887,17cyclictest7850irq/56-dwc_otg12:35:442
1619799884,10cyclictest0-21swapper/211:35:472
16195998826,44cyclictest16547-21find08:40:240
16195998819,58cyclictest0-21swapper/008:15:390
16195998818,55cyclictest0-21swapper/009:15:360
16195998724,41cyclictest16444-21gpgv08:40:240
16195998624,38cyclictest31848-21latency_hist09:55:230
16195998526,46cyclictest9356-21cut11:25:410
16195998522,54cyclictest4418-21ssh11:20:010
16195998522,40cyclictest5298-21modprobe08:10:220
1619699843,74cyclictest28499-21ssh10:30:261
16196998428,49cyclictest28371-21latency_hist10:30:241
16196998428,49cyclictest28371-21latency_hist10:30:241
16195998428,33cyclictest32675-21/usr/sbin/munin10:35:490
16195998427,42cyclictest8924-21ssh12:04:580
16195998424,12cyclictest25371-21apt09:05:230
16195998424,12cyclictest25371-21apt09:05:230
16195998417,52cyclictest8754-21modprobe11:25:230
1619799834,19cyclictest21616-21cpuspeed11:40:402
16195998323,52cyclictest0-21swapper/008:24:170
16195998320,54cyclictest0-21swapper/009:21:560
16195998243,29cyclictest8691-21/usr/sbin/munin10:05:460
16195998229,48cyclictest13622-21/usr/sbin/munin10:50:490
16195998224,41cyclictest27611-21cat07:40:420
16195998222,47cyclictest15428-21taskset10:14:380
1619599821,40cyclictest0-21swapper/007:15:350
16195998123,46cyclictest31135-21ssh11:12:050
16195998119,50cyclictest0-21swapper/009:41:240
136822810,5sleep11619699cyclictest08:30:481
16195998026,37cyclictest6528-21ssh10:42:120
16195998021,43cyclictest0-21swapper/010:25:220
16195998021,33cyclictest0-21swapper/007:35:230
16196997911,47cyclictest27289-21timerandwakeup12:25:521
16196997813,42cyclictest5269-21sh09:22:001
16195997824,44cyclictest31592-21sh10:33:590
16195997819,45cyclictest0-21swapper/007:12:210
1619699775,66cyclictest8705-21ntp_states10:05:471
1619699773,49cyclictest0-21swapper/111:50:481
16195997722,40cyclictest22479-21ssh11:01:160
16195997722,40cyclictest22479-21ssh11:01:160
16195997624,41cyclictest25881-21fschecks_count09:05:390
16195997618,39cyclictest0-21swapper/011:55:540
1619699741,58cyclictest9007-21ssh11:25:281
16195997418,41cyclictest0-21swapper/009:47:000
16195997317,47cyclictest750-21snmpd09:25:490
16195997317,42cyclictest0-21swapper/007:54:300
16195997317,42cyclictest0-21swapper/007:54:300
16195997315,41cyclictest0-21swapper/010:17:260
221822720,2sleep30-21swapper/309:42:523
16196997216,35cyclictest25573-21df_inode07:35:411
1619699721,44cyclictest22377-21df_inode12:20:401
16195997227,43cyclictest25822-21kernelversion07:35:450
16195997221,38cyclictest0-21swapper/007:45:420
16195997219,39cyclictest0-21swapper/008:51:230
16195997219,38cyclictest2057-21wc08:00:380
103152720,2sleep30-21swapper/312:05:463
307672710,3sleep20-21swapper/210:32:272
1619799711,28cyclictest0-21swapper/211:15:372
1619699711,56cyclictest28898-21sh11:49:421
1619699711,56cyclictest28898-21sh11:49:421
1619699711,51cyclictest12212-21/usr/sbin/munin09:30:431
1619699711,51cyclictest12212-21/usr/sbin/munin09:30:431
1619599711,9cyclictest7850irq/56-dwc_otg08:55:360
16195997118,40cyclictest0-21swapper/011:07:000
16195997116,46cyclictest0-21swapper/008:45:540
1619699702,52cyclictest7850irq/56-dwc_otg10:04:061
1619699701,59cyclictest17186-21ssh09:36:151
1619699701,50cyclictest7850irq/56-dwc_otg10:20:231
16195997023,35cyclictest0-21swapper/007:27:180
1619699695,47cyclictest7850irq/56-dwc_otg09:47:271
1619699691,46cyclictest17328-21/usr/sbin/munin08:40:471
16195996920,32cyclictest0-21swapper/008:26:420
1619699681,51cyclictest26420-21/usr/sbin/munin09:05:481
1619699681,50cyclictest20543-21df_inode10:20:391
16195996823,29cyclictest0-21swapper/008:34:560
16195996721,33cyclictest1402-21taskset07:56:530
1619799661,4cyclictest8050irq/56-dwc_otg_11:25:372
1619699661,48cyclictest0-21swapper/110:46:241
1619699655,39cyclictest3499-21apt08:05:221
1619699651,47cyclictest0-21swapper/109:25:391
1619699651,43cyclictest30457-21ssh09:52:541
16198996433,9cyclictest8050irq/56-dwc_otg_12:35:423
1619699642,48cyclictest15504-21ssh10:53:361
1619699642,45cyclictest8050irq/56-dwc_otg_11:20:301
1619699642,45cyclictest8050irq/56-dwc_otg_11:20:301
49122630,3sleep227174-21diskmemload11:20:312
49122630,3sleep227174-21diskmemload11:20:312
16197996310,36cyclictest0-21swapper/210:30:242
16197996310,36cyclictest0-21swapper/210:30:242
16197996310,21cyclictest23488-21kworker/2:210:05:462
1619699631,43cyclictest8260-21cron10:45:221
1619699631,36cyclictest8050irq/56-dwc_otg_11:19:391
210012620,2sleep10-21swapper/108:50:481
16197996223,24cyclictest0-21swapper/210:55:532
1619699623,37cyclictest30350-21grep11:10:511
1619699621,45cyclictest21655-21sh09:41:531
1619699621,45cyclictest1145-21ssh09:17:001
16195996214,36cyclictest0-21swapper/007:20:440
158292623,12sleep00-21swapper/007:09:080
1619699613,38cyclictest9164-21apt-get12:05:231
1619699611,45cyclictest11867-21runrttasks12:07:361
1619699611,37cyclictest0-21swapper/111:10:141
1619699603,40cyclictest0-21swapper/112:00:051
1619699601,45cyclictest10418-21chrt08:21:381
1619699601,41cyclictest0-21swapper/110:10:521
1619699601,39cyclictest22536-21chrt11:01:211
1619699601,39cyclictest22536-21chrt11:01:211
1619899599,13cyclictest750-21snmpd10:55:543
1619799593,12cyclictest30454-21rm11:50:492
1619699592,45cyclictest0-21swapper/110:39:521
1619699591,43cyclictest8082-21/usr/sbin/munin08:15:481
1619899588,24cyclictest18040-21grep11:35:453
1619899585,26cyclictest0-21swapper/310:30:243
1619899585,26cyclictest0-21swapper/310:30:243
1619699583,44cyclictest16717-21ssh12:13:511
1619699583,42cyclictest8050irq/56-dwc_otg_08:10:461
1619699581,4cyclictest14668-21/usr/sbin/munin08:35:491
1619699581,44cyclictest28932-21ssh09:11:001
1619699581,40cyclictest25554-21sort09:05:241
1619699581,40cyclictest25554-21sort09:05:241
1619699572,42cyclictest7850irq/56-dwc_otg12:31:171
1619699572,42cyclictest7850irq/56-dwc_otg12:31:171
1619699571,44cyclictest27155-21/usr/sbin/munin07:40:491
158552563,12sleep10-21swapper/107:09:291
147172560,2sleep20-21swapper/207:05:412
255652550,2sleep339-21ksoftirqd/312:24:503
1619799543,13cyclictest0-21swapper/211:30:462
1619699541,4cyclictest12473-21sshd08:29:501
1619699541,3cyclictest0-21swapper/108:45:361
1619699541,2cyclictest0-21swapper/108:57:221
16198995313,14cyclictest39-21ksoftirqd/311:30:463
1619899523,15cyclictest0-21swapper/312:15:493
1619899523,15cyclictest0-21swapper/312:15:493
1619699526,41cyclictest23391-21sed07:30:231
1619699521,38cyclictest0-21swapper/108:07:421
17242510,1sleep30-21swapper/311:15:493
1619699519,38cyclictest30822-21aten_rbpower_en07:50:381
1619699519,38cyclictest30822-21aten_rbpower_en07:50:381
1619699511,6cyclictest1305-21sed07:55:551
1619699511,2cyclictest0-21swapper/107:12:241
1619699502,34cyclictest727-21runrttasks07:17:231
16196995011,35cyclictest21459-21apt-key07:25:241
1619899493,24cyclictest0-21swapper/311:50:483
16198994828,15cyclictest28495-21ssh10:30:263
1619699483,41cyclictest29780-21sshd07:45:501
1619699481,40cyclictest7950irq/56-dwc_otg_07:30:331
1619899466,11cyclictest39-21ksoftirqd/310:05:443
1619899457,3cyclictest7950irq/56-dwc_otg_09:30:343
1619899457,3cyclictest7950irq/56-dwc_otg_09:30:343
1619899443,25cyclictest0-21swapper/309:15:423
16198994327,5cyclictest750-21snmpd09:52:493
16198994218,5cyclictest31523-21sh11:12:483
16197994220,6cyclictest25909-21sh09:47:072
1619799412,3cyclictest8050irq/56-dwc_otg_10:00:342
1619799411,4cyclictest8050irq/56-dwc_otg_09:20:342
222682400,1sleep20-21swapper/210:22:052
1619899403,20cyclictest6393-21ssh10:41:573
16198993916,11cyclictest490650ssh12:00:073
1619899390,29cyclictest8783-21ssh09:25:533
1619799394,4cyclictest8050irq/56-dwc_otg_09:10:352
1619799393,10cyclictest8050irq/56-dwc_otg_08:10:372
16198993818,5cyclictest21567-21latency_hist07:25:243
16198993817,4cyclictest22175-21if_wlan007:25:423
16198993816,10cyclictest3084050ssh09:14:313
16198993810,6cyclictest446650ssh09:20:513
1619799384,15cyclictest7950irq/56-dwc_otg_08:40:402
1619799382,3cyclictest7950irq/56-dwc_otg_09:00:342
1619799382,3cyclictest7950irq/56-dwc_otg_09:00:342
1619799381,9cyclictest19799-21ssh09:40:362
16197993817,13cyclictest29298-21latency_hist11:50:232
16197993817,13cyclictest29298-21latency_hist11:50:232
1619799381,4cyclictest0-21swapper/212:05:422
1619899378,5cyclictest7950irq/56-dwc_otg_08:10:373
16198993717,6cyclictest2733-21rt-features08:00:483
16198993711,8cyclictest0-21swapper/310:15:393
1619799379,4cyclictest31676-21ssh09:15:352
1619799375,13cyclictest7950irq/56-dwc_otg_12:30:512
1619799375,13cyclictest7950irq/56-dwc_otg_12:30:512
1619799372,24cyclictest8050irq/56-dwc_otg_11:03:292
1619799372,24cyclictest8050irq/56-dwc_otg_11:03:292
1619799371,30cyclictest0-21swapper/212:27:292
1619799371,21cyclictest0-21swapper/208:15:502
1619899363,14cyclictest8050irq/56-dwc_otg_07:15:493
1619799364,17cyclictest15386-21ssh10:53:222
1619799363,4cyclictest7950irq/56-dwc_otg_09:05:342
16197993613,5cyclictest16385-21latency_hist08:40:242
1619899356,14cyclictest16835-21ssh09:35:513
16198993514,6cyclictest6117-21expr12:00:473
16198993513,5cyclictest37-21rcuc/308:50:503
16198993512,4cyclictest26063-21grep11:05:493
1619899351,17cyclictest12826-21basename10:50:383
1619899351,15cyclictest0-21swapper/312:30:373
1619899351,15cyclictest0-21swapper/312:30:373
1619799353,4cyclictest7950irq/56-dwc_otg_12:20:372
16197993528,5cyclictest18079-21cut07:15:382
16197993516,5cyclictest29337-21sh09:50:522
16197993514,12cyclictest15040-21grep12:10:542
1619799351,25cyclictest5449-21ls08:10:232
1619799351,22cyclictest5422-21ssh10:40:482
27082340,5sleep31619899cyclictest10:37:563
1619899348,3cyclictest8050irq/56-dwc_otg_08:55:373
1619899347,24cyclictest0-21swapper/308:45:363
1619899341,22cyclictest5306-21latency_hist08:10:233
1619799347,17cyclictest0-21swapper/211:05:502
1619799347,13cyclictest8050irq/56-dwc_otg_08:01:252
1619799344,14cyclictest16990-21vmstat09:35:532
1619799341,7cyclictest0-21swapper/210:15:342
1619799341,5cyclictest7850irq/56-dwc_otg09:30:332
1619799341,5cyclictest7850irq/56-dwc_otg09:30:332
1619799341,4cyclictest0-21swapper/207:35:362
1619899339,4cyclictest0-21swapper/312:10:393
1619899339,21cyclictest0-21swapper/308:40:303
1619899337,24cyclictest0-21swapper/308:35:363
1619899336,3cyclictest0-21swapper/310:24:453
1619899331,7cyclictest6219-21unixbench_singl11:20:543
1619899331,7cyclictest6219-21unixbench_singl11:20:543
16198993317,4cyclictest7754-21if_eth008:15:423
1619899331,5cyclictest12302-21diskstats10:10:393
16198993313,13cyclictest25141-21taskset09:02:453
16198993313,13cyclictest25141-21taskset09:02:453
1619799332,13cyclictest0-21swapper/210:46:362
1619799332,10cyclictest7850irq/56-dwc_otg08:20:402
1619799331,23cyclictest0-21swapper/211:55:362
1619799331,18cyclictest0-21swapper/208:53:372
1619799331,17cyclictest0-21swapper/209:25:382
1619899328,4cyclictest0-21swapper/311:02:313
1619899328,4cyclictest0-21swapper/311:02:313
1619899324,10cyclictest0-21swapper/308:30:333
16198993222,7cyclictest505-21cron07:55:223
16198993222,7cyclictest505-21cron07:55:223
16198993215,4cyclictest381ktimersoftd/309:46:583
16198993212,4cyclictest7950irq/56-dwc_otg_11:48:103
16198993212,4cyclictest7950irq/56-dwc_otg_11:48:103
1619799325,12cyclictest8050irq/56-dwc_otg_08:45:372
1619799321,4cyclictest8050irq/56-dwc_otg_07:10:512
16197993212,13cyclictest570-21grep10:35:402
1619899317,11cyclictest7950irq/56-dwc_otg_07:30:373
1619899317,10cyclictest39-21ksoftirqd/310:00:413
1619899316,4cyclictest7850irq/56-dwc_otg10:45:463
1619899311,12cyclictest8050irq/56-dwc_otg_12:30:233
1619799318,4cyclictest1219050wget10:10:372
1619799316,10cyclictest0-21swapper/207:45:472
1619799315,5cyclictest7950irq/56-dwc_otg_08:25:502
1619799315,4cyclictest0-21swapper/212:03:382
1619799312,14cyclictest8050irq/56-dwc_otg_11:10:442
1619799311,21cyclictest24051-21ls07:30:452
1619899305,14cyclictest7950irq/56-dwc_otg_07:45:363
16198993011,5cyclictest285532chrt07:43:243
1619799306,21cyclictest14130-21kworker/u8:008:30:332
1619799302,16cyclictest0-21swapper/207:20:472
1619899297,15cyclictest0-21swapper/309:05:363
1619899294,7cyclictest8050irq/56-dwc_otg_08:25:533
1619799291,17cyclictest31-21ksoftirqd/207:55:442
1619799291,11cyclictest0-21swapper/207:40:442
16198992821,5cyclictest17773-21sh07:15:223
1619799288,4cyclictest8050irq/56-dwc_otg_07:50:382
1619799288,4cyclictest8050irq/56-dwc_otg_07:50:382
1619799281,19cyclictest0-21swapper/207:27:002
17952270,2sleep30-21swapper/308:00:233
1619899273,12cyclictest7850irq/56-dwc_otg08:20:403
1619899271,4cyclictest75050snmpd07:38:363
1619799271,13cyclictest0-21swapper/208:57:322
156212271,16sleep30-21swapper/307:06:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional