You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-21 - 05:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Broadcom BCM2835 @1400 MHz, Linux 4.18.16-rt9 (Profile)

Latency plot of shadow in rack #7, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Mar 21, 2023 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
575122830,4sleep34150-21kworker/3:119:06:513
6381991036,25cyclictest0-21swapper/020:41:260
638199927,61cyclictest0-21swapper/019:36:400
638199926,69cyclictest0-21swapper/020:36:380
638499813,38cyclictest8050irq/56-dwc_otg_22:16:393
138582810,3sleep113855-21sh21:51:231
638199801,45cyclictest0-21swapper/022:11:370
6381997948,10cyclictest101ktimersoftd/021:51:520
638199747,45cyclictest0-21swapper/020:06:460
6381997413,18cyclictest0-21swapper/021:36:410
638499722,49cyclictest0-21swapper/320:41:253
6381997044,15cyclictest8050irq/56-dwc_otg_23:31:380
69872690,2sleep30-21swapper/323:56:123
6381996850,10cyclictest101ktimersoftd/023:11:200
6381996850,10cyclictest101ktimersoftd/023:11:200
638199680,40cyclictest0-21swapper/000:11:330
638399677,27cyclictest0-21swapper/221:21:382
638399674,45cyclictest8050irq/56-dwc_otg_23:42:002
638399674,45cyclictest8050irq/56-dwc_otg_23:42:002
638399671,45cyclictest28473-21ssh22:56:382
638199678,40cyclictest0-21swapper/019:21:360
638199678,40cyclictest0-21swapper/019:21:360
638399661,52cyclictest0-21swapper/221:10:112
638199664,52cyclictest13-21rcuc/000:26:510
638199662,10cyclictest0-21swapper/022:41:380
6381996516,6cyclictest15211-21idleruntime-cro19:36:200
638399644,49cyclictest12222-21seq20:56:392
638399643,49cyclictest23196-21ssh23:32:072
638399642,50cyclictest18324-21ssh23:26:432
638399642,42cyclictest7850irq/56-dwc_otg19:26:432
638399641,52cyclictest1581-21ssh23:47:252
6381996435,20cyclictest0-21swapper/021:25:460
638199641,4cyclictest17426-21ssh22:41:210
638199641,48cyclictest17201-21munin-run21:11:200
638199641,45cyclictest18725-21ssh21:11:500
638399633,45cyclictest18740-21unixbench_singl21:56:522
638399631,4cyclictest32282-21cpu23:01:412
638199636,41cyclictest9-21ksoftirqd/023:46:550
6381996329,12cyclictest101ktimersoftd/020:51:340
638199631,47cyclictest7498-21aten_rbpower_po23:56:370
638199631,31cyclictest31041-21grep20:16:520
638399622,36cyclictest8050irq/56-dwc_otg_22:35:562
638399621,39cyclictest17116-21ssh22:41:012
638199621,46cyclictest17369-21aten_rbpower_vo19:41:350
638399617,39cyclictest10788-21ps19:21:482
638399617,39cyclictest10788-21ps19:21:482
638399613,47cyclictest10497-21grep20:51:402
638399612,43cyclictest0-21swapper/221:51:282
638399611,47cyclictest2216-21ssh22:20:372
638399611,40cyclictest7184-21if_wlan021:41:452
638199611,46cyclictest0-21swapper/000:36:580
638399602,12cyclictest7950irq/56-dwc_otg_20:16:342
638399601,45cyclictest0-21swapper/200:10:132
638399601,44cyclictest0-21swapper/200:31:542
638399601,43cyclictest0-21swapper/222:06:312
638399601,38cyclictest3597-21diskstats23:06:452
638399601,38cyclictest3597-21diskstats23:06:452
638199602,47cyclictest0-21swapper/000:31:460
638199601,9cyclictest32421-21cut23:01:440
638399592,47cyclictest7950irq/56-dwc_otg_22:13:102
638399591,45cyclictest0-21swapper/222:30:502
638399591,42cyclictest14342-21aten_rbpower_en23:21:382
638399591,40cyclictest8050irq/56-dwc_otg_22:51:202
638399591,3cyclictest0-21swapper/223:58:052
638199595,42cyclictest9-21ksoftirqd/019:21:210
638199595,28cyclictest15237-21ntpq00:06:540
6381995941,12cyclictest101ktimersoftd/023:41:500
6381995941,12cyclictest101ktimersoftd/023:41:500
638199592,27cyclictest25848-21grep00:21:570
638399583,42cyclictest1598-21ssh21:33:412
638399583,41cyclictest17515-21modprobe22:41:212
638399583,37cyclictest7850irq/56-dwc_otg23:16:532
638399582,3cyclictest8050irq/56-dwc_otg_23:11:512
6383995816,29cyclictest2322-21sh00:36:252
638399581,47cyclictest17193-21diskmemload21:12:362
638399581,42cyclictest19613-21ssh00:13:072
638299582,10cyclictest7850irq/56-dwc_otg20:01:351
638199589,30cyclictest18994-21snmpd21:26:400
638199586,43cyclictest0-21swapper/022:46:350
6381995839,10cyclictest101ktimersoftd/021:01:480
6381995833,8cyclictest32558-21grep20:21:510
6381995832,7cyclictest101ktimersoftd/019:26:370
6381995826,15cyclictest7950irq/56-dwc_otg_22:22:000
638199582,42cyclictest0-21swapper/023:16:530
638199582,33cyclictest0-21swapper/022:06:530
638199581,6cyclictest7477-21ntp_states21:41:500
638199581,45cyclictest21384-21grep22:01:410
638199581,35cyclictest0-21swapper/022:31:400
638399571,46cyclictest18994-21snmpd21:36:362
638399571,43cyclictest5904-21runrttasks22:25:432
638399571,43cyclictest23001-21ssh00:18:182
638399571,3cyclictest7950irq/56-dwc_otg_19:11:332
638199571,42cyclictest25571-21tr23:36:420
638399552,36cyclictest8050irq/56-dwc_otg_23:36:562
638399552,34cyclictest8050irq/56-dwc_otg_21:51:212
638399552,34cyclictest8050irq/56-dwc_otg_21:51:212
638399551,44cyclictest29997-21taskset00:28:402
638399551,43cyclictest8347-21sed20:46:202
6381995532,9cyclictest101ktimersoftd/023:01:200
638199551,42cyclictest0-21swapper/023:11:500
638399543,38cyclictest8050irq/56-dwc_otg_00:26:052
638399542,49cyclictest21160-21ssh21:16:352
638399541,4cyclictest8430-21df_inode19:16:372
638399541,4cyclictest22779-21/usr/sbin/munin19:56:362
638399541,39cyclictest7850irq/56-dwc_otg21:30:152
638399541,36cyclictest0-21swapper/219:52:382
638199543,31cyclictest0-21swapper/021:46:340
638199543,31cyclictest0-21swapper/021:46:340
6381995429,10cyclictest101ktimersoftd/020:16:200
6381995429,10cyclictest101ktimersoftd/020:16:200
638199542,39cyclictest0-21swapper/000:16:450
6381995422,10cyclictest12097-21cut20:56:350
638199541,31cyclictest21200-21wget21:16:380
176232540,2sleep30-21swapper/319:41:393
638499533,20cyclictest28127-21taskset22:11:373
638399531,4cyclictest29060-21ls20:11:502
638399531,4cyclictest29060-21ls20:11:502
638399531,42cyclictest5210-21ssh23:52:252
638399531,37cyclictest10095-21wc20:51:202
638199536,25cyclictest32467-21grep22:16:500
638199531,34cyclictest0-21swapper/019:46:470
638399522,3cyclictest0-21swapper/222:53:362
638399522,39cyclictest8050irq/56-dwc_otg_20:24:352
638399521,6cyclictest15499-21aten_rbpower_en19:36:402
638399521,48cyclictest13927-21grep21:01:372
638299527,29cyclictest0-21swapper/119:36:401
638199523,6cyclictest0-21swapper/020:31:200
6381995232,10cyclictest101ktimersoftd/000:05:110
6381995226,9cyclictest18118-21aten_rbpower_cu23:26:390
6383995122,27cyclictest12046-21switchtime00:01:572
638399511,46cyclictest25552-21grep20:01:512
638399511,37cyclictest8050irq/56-dwc_otg_22:02:242
638299518,28cyclictest0-21swapper/120:26:371
638299516,15cyclictest7415-21kernelversion23:11:441
638199518,4cyclictest0-21swapper/020:46:510
6381995128,5cyclictest101ktimersoftd/023:24:050
6381995126,11cyclictest101ktimersoftd/022:53:460
638199511,27cyclictest2883-21latency_hist21:36:200
638399501,5cyclictest5242-21memory20:36:452
638399501,5cyclictest13846-21/usr/sbin/munin19:31:362
638399501,40cyclictest18994-21snmpd20:30:172
638299507,25cyclictest0-21swapper/119:21:361
638299507,25cyclictest0-21swapper/119:21:361
6381995037,4cyclictest4052-21perl23:51:430
638199502,28cyclictest0-21swapper/019:11:340
6381995010,14cyclictest8050irq/56-dwc_otg_20:01:500
638399492,43cyclictest3574-21/usr/sbin/munin20:31:462
638299498,11cyclictest1862-21ssh21:34:161
6382994923,10cyclictest8050irq/56-dwc_otg_21:46:421
6382994923,10cyclictest8050irq/56-dwc_otg_21:46:421
6381994932,4cyclictest101ktimersoftd/019:57:470
638499488,25cyclictest0-21swapper/300:11:363
638399481,5cyclictest18942-21date19:46:202
638399481,4cyclictest19757-21ntp_states19:46:452
638399481,44cyclictest26534-21fschecks_count20:06:462
638299487,22cyclictest0-21swapper/119:16:201
6382994825,12cyclictest7950irq/56-dwc_otg_00:04:561
6382994824,9cyclictest8050irq/56-dwc_otg_19:31:501
638499478,26cyclictest0-21swapper/319:16:353
638499471,5cyclictest7950irq/56-dwc_otg_23:31:373
638299479,22cyclictest26700-21ssh21:23:261
638299477,27cyclictest0-21swapper/119:56:351
638299474,5cyclictest7950irq/56-dwc_otg_23:56:341
6382994714,11cyclictest7950irq/56-dwc_otg_23:01:211
638199471,33cyclictest9312-21taskset22:30:070
638499464,4cyclictest8050irq/56-dwc_otg_21:36:383
638299469,21cyclictest30095-21ls23:41:561
638299469,21cyclictest30095-21ls23:41:561
638299469,20cyclictest29979-21ssh00:28:371
638299468,23cyclictest20409-21ssh21:15:251
638299467,24cyclictest0-21swapper/100:31:371
638299466,13cyclictest28122-21latency_hist22:56:201
638299465,23cyclictest20484-21ssh00:15:191
638299463,4cyclictest7950irq/56-dwc_otg_23:36:391
6382994625,8cyclictest8050irq/56-dwc_otg_22:43:501
638299462,13cyclictest0-21swapper/121:56:341
6382994620,5cyclictest20689-21latency_hist19:51:201
6382994612,23cyclictest14284-21sh23:21:351
59532462,13sleep00-21swapper/019:09:160
638499457,24cyclictest0-21swapper/321:06:393
638499457,24cyclictest0-21swapper/320:56:343
638299457,22cyclictest0-21swapper/120:36:381
638299455,22cyclictest30792-21ls20:16:491
6381994526,4cyclictest101ktimersoftd/021:57:030
6381994514,14cyclictest3434-21/usr/sbin/munin20:31:430
638299449,16cyclictest12054-21/usr/sbin/munin19:26:391
638299448,23cyclictest0-21swapper/119:51:371
638299447,25cyclictest0-21swapper/123:06:411
638299447,25cyclictest0-21swapper/123:06:411
638299445,17cyclictest11190-21ssh23:16:441
6382994411,15cyclictest23274-21ssh22:48:391
638299438,22cyclictest0-21swapper/120:21:381
638299438,15cyclictest22172-21fschecks_time23:31:461
638299437,22cyclictest12123-21cpu20:56:361
638299435,24cyclictest13943-21ssh22:36:411
6382994321,4cyclictest7850irq/56-dwc_otg23:01:391
6382994316,12cyclictest8050irq/56-dwc_otg_23:26:401
638499427,25cyclictest0-21swapper/323:56:353
6384994226,9cyclictest7834-21ssh23:11:503
6384994216,12cyclictest0-21swapper/321:31:343
638299427,11cyclictest8280-21aten_rbpower_po19:16:361
638299426,9cyclictest3677-21ssh23:51:351
638299423,22cyclictest11003-21/usr/sbin/munin22:31:531
6382994225,8cyclictest7950irq/56-dwc_otg_22:10:281
6382994223,7cyclictest7850irq/56-dwc_otg22:02:201
638299422,24cyclictest7282-21ssh22:26:481
6382994217,11cyclictest7850irq/56-dwc_otg21:41:431
6384994112,7cyclictest0-21swapper/319:56:393
6382994119,8cyclictest7850irq/56-dwc_otg22:15:451
638199416,22cyclictest0-21swapper/019:51:370
110212410,5sleep3638499cyclictest00:01:273
6384994012,4cyclictest0-21swapper/320:21:393
6382994020,8cyclictest8050irq/56-dwc_otg_22:23:081
6382994018,8cyclictest7850irq/56-dwc_otg00:41:181
6382994012,15cyclictest8050irq/56-dwc_otg_19:42:161
6382994012,11cyclictest30438-21ssh21:28:431
638499398,18cyclictest39-21ksoftirqd/323:26:383
638499391,4cyclictest7850irq/56-dwc_otg23:41:373
638499391,4cyclictest7850irq/56-dwc_otg23:41:373
638299393,21cyclictest0-21swapper/121:36:391
6382993913,6cyclictest221ktimersoftd/120:51:371
638299390,21cyclictest16841-21taskset21:07:141
6384993817,5cyclictest24803-21/usr/sbin/munin22:51:513
638499381,25cyclictest8050irq/56-dwc_otg_20:41:213
638299389,6cyclictest0-21swapper/123:46:361
638299380,11cyclictest22871-21ssh00:17:551
638499376,7cyclictest37-21rcuc/323:46:563
638499372,5cyclictest8050irq/56-dwc_otg_19:31:343
638499372,4cyclictest7950irq/56-dwc_otg_21:01:343
638499372,23cyclictest39-21ksoftirqd/321:51:373
638499371,6cyclictest24284-21awk22:06:203
638499371,5cyclictest0-21swapper/300:31:353
638499371,25cyclictest0-21swapper/320:11:373
638499371,25cyclictest0-21swapper/320:11:373
638499371,21cyclictest7950irq/56-dwc_otg_23:01:213
638299379,11cyclictest17035-21ssh00:10:091
638299377,18cyclictest23991-21ssh21:20:311
6382993718,4cyclictest7850irq/56-dwc_otg00:23:061
638299368,14cyclictest0-21swapper/120:51:171
638299363,18cyclictest27802-21taskset20:11:091
6382993611,22cyclictest0-21swapper/121:01:361
6384993516,4cyclictest39-21ksoftirqd/323:36:473
638499351,4cyclictest14312-21chrt22:36:463
6384993510,14cyclictest0-21swapper/323:25:183
638299357,13cyclictest29450-21taskset20:14:031
638299357,13cyclictest29450-21taskset20:14:031
638299354,7cyclictest429-21grep22:16:561
58522353,23sleep10-21swapper/119:07:591
638499347,6cyclictest8050irq/56-dwc_otg_23:01:443
638499346,19cyclictest6734-21ssh21:41:393
6384993422,6cyclictest26415-21ssh00:23:183
638499342,17cyclictest0-21swapper/322:00:053
638499341,8cyclictest15148-21grep00:06:523
638499341,5cyclictest8050irq/56-dwc_otg_00:16:343
638499341,17cyclictest12834-21grep19:26:513
6384993411,12cyclictest0-21swapper/322:21:563
6384993410,5cyclictest0-21swapper/322:26:423
45832349,13sleep20-21swapper/219:06:352
267552340,2sleep30-21swapper/322:08:563
638499337,5cyclictest8050irq/56-dwc_otg_20:06:463
638499332,10cyclictest8050irq/56-dwc_otg_23:16:413
6384993314,5cyclictest39-21ksoftirqd/321:46:473
6384993314,5cyclictest39-21ksoftirqd/321:46:473
6382993315,7cyclictest7850irq/56-dwc_otg20:35:141
638299330,18cyclictest6558-21gdbus20:41:211
638499324,16cyclictest19051-21/usr/sbin/munin19:46:473
638499323,16cyclictest0-21swapper/322:43:053
638499321,9cyclictest3430-21wc00:36:563
6384993218,5cyclictest26295-21ssh21:22:333
638499321,16cyclictest22535-21sh22:46:573
638499321,12cyclictest8050irq/56-dwc_otg_00:26:553
638499319,11cyclictest6380-21cyclictest22:32:003
638499316,14cyclictest0-21swapper/320:46:363
638499314,4cyclictest8050irq/56-dwc_otg_20:26:533
638499312,9cyclictest28553-21diskstats21:26:393
638499311,6cyclictest21259-21aten_rbpower_po21:16:393
638499311,4cyclictest30091-21df20:16:383
638499311,26cyclictest24730-21cut20:01:403
638499311,12cyclictest37-21rcuc/320:54:093
6384993110,12cyclictest17855-21tune2fs21:11:383
51362310,4sleep3638499cyclictest23:08:013
51362310,4sleep3638499cyclictest23:08:013
638499301,21cyclictest0-21swapper/319:21:403
638499301,21cyclictest0-21swapper/319:21:403
638499296,14cyclictest0-21swapper/319:11:363
638499283,12cyclictest0-21swapper/319:51:483
638499281,20cyclictest0-21swapper/319:36:443
638499263,15cyclictest0-21swapper/320:35:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional