You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 00:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Sat Sep 19, 2026 12:43:40)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2176621220,3sleep1510399cyclictest07:55:421
1876321150,12sleep27850irq/56-dwc_otg_10:35:442
1356821120,5sleep3510599cyclictest12:32:483
51029910954,28cyclictest0-21swapper/007:10:250
51039910034,19cyclictest0-21swapper/107:10:251
267032940,3sleep30-21swapper/312:08:223
94352920,2sleep20-21swapper/210:24:032
136472840,6sleep0510299cyclictest09:47:380
16692810,2sleep00-21swapper/009:32:110
510299767,25cyclictest0-21swapper/009:50:360
5102997619,43cyclictest20522-21grep09:55:560
5102997614,58cyclictest4259-21latency_hist11:40:240
232812740,3sleep30-21swapper/311:22:153
510299722,59cyclictest0-21swapper/011:05:450
5102997215,9cyclictest0-21swapper/008:35:350
5103997128,22cyclictest8677-21ssh10:22:331
5103997041,9cyclictest19676-21apt-get07:50:371
5102997023,36cyclictest1611-21sh10:55:390
510299701,56cyclictest12561-21head12:30:540
5102997011,49cyclictest0-21swapper/009:25:380
5103996925,23cyclictest16018-21modprobe11:55:241
5103996924,26cyclictest3024050ssh12:12:371
510299694,55cyclictest0-21swapper/010:05:530
510299692,5cyclictest13908-21cron07:35:230
204062690,2sleep20-21swapper/209:55:542
5103996825,19cyclictest18211-21sed10:35:241
5103996824,25cyclictest17225-21ssh12:37:161
510299684,47cyclictest30288-21munin-run08:20:230
5102996712,40cyclictest0-21swapper/007:55:540
5103996627,14cyclictest5959-21ssh10:19:561
510299668,18cyclictest0-21swapper/012:35:390
5103996526,22cyclictest2942-21ssh11:38:001
5103996526,12cyclictest60-21kswapd008:20:301
5103996524,24cyclictest31824-21ssh10:51:421
5103996523,22cyclictest14111-21latency_hist07:35:241
5103996522,25cyclictest8655-21taskset08:49:021
510299658,30cyclictest26578-21fschecks_time10:45:420
510399648,29cyclictest0-21swapper/109:00:291
5103996429,19cyclictest22549-21ssh12:02:511
5103996423,16cyclictest12309-21ps09:45:491
510299646,41cyclictest13-21rcuc/010:50:550
510299641,50cyclictest21283-21ls12:00:540
5103996326,24cyclictest14583-21ssh12:34:491
5103996326,16cyclictest12541-21ssh11:09:301
5103996324,27cyclictest31555-21sh09:29:521
510299633,44cyclictest0-21swapper/010:00:400
510299632,34cyclictest11163-21latency_hist12:30:240
510399624,9cyclictest7850irq/56-dwc_otg_09:55:391
5103996227,13cyclictest22309-21ssh10:40:381
5103996226,13cyclictest26528-21seq08:08:591
5103996226,11cyclictest0-21swapper/111:40:311
5103996224,13cyclictest24754-21ssh11:25:111
5103996223,22cyclictest16007-21seq09:09:301
5103996223,20cyclictest8538-21runrttasks11:04:201
5103996222,13cyclictest18808-21ssh10:35:451
5103996219,30cyclictest31292-21sh11:32:381
5102996232,6cyclictest7950irq/56-dwc_otg_09:10:230
510299622,52cyclictest16065-21kworker/0:209:10:260
510299622,52cyclictest16065-21kworker/0:209:10:260
510299621,41cyclictest14621-21unixbench_singl11:10:560
510299621,26cyclictest20988-21mktemp11:20:240
510299621,26cyclictest20988-21mktemp11:20:240
510399614,7cyclictest7750irq/56-dwc_otg07:20:371
510399614,7cyclictest7750irq/56-dwc_otg07:20:371
5103996127,21cyclictest17347-21seq07:41:591
5103996125,19cyclictest9031-21ssh12:26:321
5103996125,14cyclictest26201-21ssh12:07:221
5103996124,22cyclictest4513-21ssh10:59:061
5103996120,19cyclictest20683-21ssh11:20:051
5103996120,19cyclictest20683-21ssh11:20:051
510299613,42cyclictest3105-21if_wlan010:15:450
510299612,44cyclictest0-21swapper/008:10:420
510299612,44cyclictest0-21swapper/008:10:420
510299612,31cyclictest0-21swapper/008:35:230
5105996024,22cyclictest0-21swapper/307:10:263
5103996029,19cyclictest5928-21ssh09:37:511
5103996029,19cyclictest12748-21sh10:27:521
5103996025,18cyclictest1888-21ssh10:14:401
5103996025,18cyclictest1888-21ssh10:14:401
5103996025,14cyclictest27174-21sed10:45:511
5103996022,27cyclictest18693-21ssh11:57:581
5103996019,10cyclictest8503-21apt-get11:45:351
5103996018,15cyclictest5416-21ssh12:22:091
5102996045,4cyclictest38822chrt09:35:410
5102996032,8cyclictest7950irq/56-dwc_otg_08:05:380
510299602,39cyclictest28351-21fschecks_count12:10:430
510299602,33cyclictest0-21swapper/011:45:240
5102996018,27cyclictest0-21swapper/011:30:540
510299601,44cyclictest0-21swapper/008:00:330
5103995927,20cyclictest19402-21ssh09:14:101
5103995927,20cyclictest19402-21ssh09:14:101
5103995927,14cyclictest12274-21taskset08:59:121
5103995926,21cyclictest9852-21ssh09:42:521
510299597,42cyclictest1180-21runrttasks11:04:090
510299592,4cyclictest0-21swapper/010:30:550
510299591,7cyclictest14041-21awk10:30:240
510299591,46cyclictest16430-21ls07:40:420
510299591,11cyclictest30686-21apt-get10:10:310
510299591,11cyclictest30686-21apt-get10:10:310
173052590,3sleep117306-21diskmemload09:52:061
5103995824,14cyclictest456-21ssh12:15:511
510299587,29cyclictest25640-21grep09:20:550
510299582,39cyclictest0-21swapper/011:20:390
510299581,7cyclictest17634-21sed11:55:560
510599579,29cyclictest0-21swapper/309:15:403
510399576,9cyclictest7750irq/56-dwc_otg08:35:361
5103995728,12cyclictest27435-21ssh11:27:441
5103995728,12cyclictest27435-21ssh11:27:441
5103995725,17cyclictest26214-21sh10:04:181
5103995712,22cyclictest7950irq/56-dwc_otg_09:35:001
5102995719,25cyclictest0-21swapper/007:47:510
510299571,39cyclictest0-21swapper/009:40:490
5102995712,10cyclictest0-21swapper/008:45:550
510399563,9cyclictest7750irq/56-dwc_otg10:05:361
5103995626,12cyclictest28141-21taskset08:11:381
5103995626,12cyclictest28141-21taskset08:11:381
5103995624,13cyclictest22195-21ssh09:16:551
5103995624,13cyclictest15426-21ssh11:12:271
5103995623,15cyclictest953-21ls08:25:521
5103995623,15cyclictest953-21ls08:25:521
510299562,8cyclictest1030-21unixbench-2d08:25:530
510299562,8cyclictest1030-21unixbench-2d08:25:530
5102995622,5cyclictest32087-21aten_rbpower_po12:15:380
510299561,39cyclictest1180-21runrttasks11:25:460
510299561,39cyclictest1180-21runrttasks11:25:460
5102995613,17cyclictest0-21swapper/010:42:160
5103995517,22cyclictest6693-21taskset08:42:071
5102995529,12cyclictest0-21swapper/009:18:000
5102995522,19cyclictest7950irq/56-dwc_otg_11:45:390
510599547,30cyclictest0-21swapper/308:50:373
510299541,5cyclictest31384-21grep08:20:510
5102995411,23cyclictest0-21swapper/011:50:540
510599538,29cyclictest0-21swapper/309:00:313
510399538,29cyclictest0-21swapper/108:00:361
5103995323,12cyclictest27661-21ssh09:24:551
5103995322,14cyclictest8536-21sed07:20:241
5103995317,14cyclictest29754-21grep08:15:531
510299536,5cyclictest7205-21latency_hist12:25:250
510299532,35cyclictest0-21swapper/007:35:260
510299532,35cyclictest0-21swapper/007:35:260
5102995316,18cyclictest0-21swapper/007:15:510
5102995315,27cyclictest5615-21df08:40:420
510299531,29cyclictest24898-21ssh12:05:470
5103995222,18cyclictest10493-21taskset08:54:291
510299527,22cyclictest0-21swapper/010:20:460
5102995220,15cyclictest20561-21switchtime07:50:520
5103995123,13cyclictest19111-21chrt07:46:271
5103995116,14cyclictest1180-21runrttasks08:34:331
5102995112,6cyclictest21179-21sh10:38:530
510599501,16cyclictest0-21swapper/309:05:523
510399508,31cyclictest0-21swapper/107:35:371
510399508,31cyclictest0-21swapper/107:35:371
5103995015,18cyclictest0-21swapper/107:28:581
510299502,33cyclictest0-21swapper/007:20:510
510299502,33cyclictest0-21swapper/007:20:510
47322502,38sleep00-21swapper/007:09:090
47322502,38sleep00-21swapper/007:09:090
510599492,14cyclictest8506-21ssh11:45:353
510499493,24cyclictest0-21swapper/207:10:262
510599489,28cyclictest0-21swapper/307:35:293
510599489,28cyclictest0-21swapper/307:35:293
510299482,36cyclictest0-21swapper/007:27:400
510299482,24cyclictest0-21swapper/009:00:520
288742480,2sleep20-21swapper/212:10:502
97522470,2sleep20-21swapper/209:42:402
314602470,2sleep20-21swapper/211:32:582
510599461,30cyclictest0-21swapper/308:25:243
510499464,6cyclictest0-21swapper/212:25:432
510499462,9cyclictest0-21swapper/208:55:392
510499462,14cyclictest7750irq/56-dwc_otg08:45:382
5104994615,9cyclictest0-21swapper/211:40:372
5102994515,20cyclictest0-21swapper/008:52:000
510599448,30cyclictest0-21swapper/307:15:373
510599444,5cyclictest0-21swapper/307:45:363
5105994426,15cyclictest2445350wget09:20:403
5105994422,10cyclictest22029-21ssh10:40:243
510499445,7cyclictest7950irq/56-dwc_otg_09:20:242
5104994430,10cyclictest7950irq/56-dwc_otg_11:10:382
510499442,6cyclictest7950irq/56-dwc_otg_12:35:362
5104994419,5cyclictest8730-21cpuspeed11:45:392
5104994416,7cyclictest301ktimersoftd/207:50:382
510299442,31cyclictest0-21swapper/008:55:310
5105994318,15cyclictest1614-21ssh10:55:403
5104994336,5cyclictest16106-21diskmemload10:25:352
510499432,8cyclictest0-21swapper/210:55:382
5104994320,13cyclictest31-21ksoftirqd/208:15:512
40312437,25sleep14106-21/usr/sbin/munin07:05:501
40312437,25sleep14106-21/usr/sbin/munin07:05:501
232302431,36sleep323236-21ssh10:00:403
5105994218,6cyclictest19312-21sh09:13:593
5105994218,6cyclictest19312-21sh09:13:593
5105994216,5cyclictest19917-21idleruntime-cro12:00:233
510499428,16cyclictest0-21swapper/210:45:382
510499422,5cyclictest0-21swapper/208:40:372
5104994220,9cyclictest7850irq/56-dwc_otg_11:55:112
510599418,18cyclictest0-21swapper/310:50:383
510599416,5cyclictest7750irq/56-dwc_otg11:25:453
510599416,5cyclictest7750irq/56-dwc_otg11:25:453
510599416,25cyclictest37-21rcuc/311:35:393
510599413,21cyclictest0-21swapper/307:23:393
510599413,21cyclictest0-21swapper/307:23:393
510599412,8cyclictest19552-21diskstats09:55:423
5105994119,6cyclictest9787-21fschecks_time11:05:423
510499414,8cyclictest0-21swapper/208:50:352
5104994116,9cyclictest31-21ksoftirqd/209:45:472
5104994114,14cyclictest29-21rcuc/208:34:342
510599408,18cyclictest0-21swapper/312:25:383
510599405,5cyclictest0-21swapper/307:35:243
5105994012,18cyclictest0-21swapper/311:55:163
510499408,6cyclictest7950irq/56-dwc_otg_10:30:392
5104994016,5cyclictest22805-21sh12:03:222
510499401,25cyclictest7750irq/56-dwc_otg12:05:542
510599398,23cyclictest7850irq/56-dwc_otg_09:50:383
510599398,23cyclictest0-21swapper/311:10:393
510499399,17cyclictest301ktimersoftd/211:27:242
510499399,17cyclictest301ktimersoftd/211:27:242
510499394,19cyclictest31-21ksoftirqd/208:20:462
510499393,5cyclictest7750irq/56-dwc_otg09:50:352
510499392,7cyclictest1646550ssh09:10:372
510499392,7cyclictest1646550ssh09:10:372
5104993920,5cyclictest3840-21fschecks_time12:20:432
510499391,28cyclictest32687-21cut08:25:412
510499391,28cyclictest32687-21cut08:25:412
510499391,28cyclictest0-21swapper/210:54:322
187152390,6sleep2510499cyclictest07:45:522
510599388,21cyclictest7950irq/56-dwc_otg_12:15:363
510599388,16cyclictest0-21swapper/310:25:373
5105993826,5cyclictest16015-21ssh10:31:233
510599382,20cyclictest0-21swapper/312:13:563
510599382,19cyclictest7850irq/56-dwc_otg_09:49:353
510599381,24cyclictest19911-21cut07:50:433
510499388,18cyclictest7950irq/56-dwc_otg_11:57:332
510499386,14cyclictest31-21ksoftirqd/211:07:122
510499382,9cyclictest3654-21ssh09:35:392
510499382,29cyclictest7950irq/56-dwc_otg_08:35:362
5104993815,12cyclictest301ktimersoftd/210:40:452
510599371,27cyclictest18197-21ssh11:15:523
510599371,27cyclictest18197-21ssh11:15:523
510599371,25cyclictest4621-21ls09:35:523
510599371,25cyclictest0-21swapper/310:22:583
5105993710,19cyclictest0-21swapper/310:16:153
510499375,6cyclictest9550irq/80-mmc010:10:382
510499375,6cyclictest9550irq/80-mmc010:10:382
510499374,7cyclictest7950irq/56-dwc_otg_07:40:382
510499373,20cyclictest0-21swapper/211:00:542
510499372,20cyclictest0-21swapper/212:19:402
5104993713,11cyclictest7950irq/56-dwc_otg_10:00:472
510499371,29cyclictest17647-21ls11:15:432
510499371,29cyclictest17647-21ls11:15:432
5104993710,17cyclictest7950irq/56-dwc_otg_10:08:242
510599366,4cyclictest0-21swapper/311:40:463
510599362,13cyclictest22563-21sed07:55:543
5105993613,6cyclictest4099-21/usr/sbin/munin12:20:463
510599361,24cyclictest0-21swapper/309:40:533
5105993611,13cyclictest0-21swapper/310:15:213
5105993611,13cyclictest0-21swapper/310:15:213
5105993611,11cyclictest0-21swapper/310:43:483
510499369,9cyclictest7750irq/56-dwc_otg07:30:512
510499365,14cyclictest7950irq/56-dwc_otg_09:20:532
510499364,4cyclictest7950irq/56-dwc_otg_08:10:382
510499364,4cyclictest7950irq/56-dwc_otg_08:10:382
510499362,19cyclictest0-21swapper/211:35:532
5104993613,8cyclictest0-21swapper/209:25:372
510499361,21cyclictest21700-21ls11:20:442
48812365,21sleep24984-21latency_hist07:10:242
48812365,21sleep24984-21latency_hist07:10:242
510599359,4cyclictest0-21swapper/312:01:563
510599358,5cyclictest7750irq/56-dwc_otg08:10:423
510599358,5cyclictest7750irq/56-dwc_otg08:10:423
510599354,5cyclictest0-21swapper/308:05:353
5105993522,6cyclictest29365-21df_inode11:30:433
510599352,18cyclictest0-21swapper/308:36:083
510599351,20cyclictest8060-21switchtime08:45:523
5105993511,13cyclictest0-21swapper/312:40:033
510499358,17cyclictest0-21swapper/209:00:312
510499354,7cyclictest7950irq/56-dwc_otg_09:05:422
510599347,15cyclictest31902-21ls09:30:243
510599342,25cyclictest0-21swapper/310:49:413
510599342,17cyclictest0-21swapper/307:40:313
510599341,30cyclictest2373-21ls08:30:463
5105993411,14cyclictest6143-21missed_timers11:00:463
510499342,7cyclictest7950irq/56-dwc_otg_08:05:442
510499341,7cyclictest7950irq/56-dwc_otg_07:25:392
5104993414,10cyclictest29-21rcuc/209:34:472
510499341,23cyclictest2818-21ssh10:15:412
510599336,12cyclictest7850irq/56-dwc_otg_08:40:403
510599331,6cyclictest28332-21ssh10:05:533
510499335,16cyclictest7950irq/56-dwc_otg_12:30:502
510499332,19cyclictest0-21swapper/207:55:332
510599322,15cyclictest0-21swapper/308:03:073
510599321,11cyclictest0-21swapper/308:26:483
510599321,11cyclictest0-21swapper/308:26:483
5105993210,8cyclictest0-21swapper/309:30:383
510499328,21cyclictest0-21swapper/208:00:362
510499324,14cyclictest29-21rcuc/207:15:482
510499323,13cyclictest7950irq/56-dwc_otg_07:20:472
510499323,13cyclictest7950irq/56-dwc_otg_07:20:472
510599316,7cyclictest30284-21apt-get08:20:233
510599315,6cyclictest11039-21diskstats08:55:423
510599308,19cyclictest0-21swapper/307:25:393
510499304,14cyclictest7950irq/56-dwc_otg_07:35:462
510499304,14cyclictest7950irq/56-dwc_otg_07:35:462
33842283,17sleep33488-21df07:05:403
33842283,17sleep33488-21df07:05:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional