You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-23 - 05:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Celeron 33455 @1500 MHz, Linux 5.15.2-rt19 (Profile)

Latency plot of shadow in rack #c, slot #7
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Thu Mar 23, 2023 00:44:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12492029913139,85cyclictest1753-21gnome-shell00:12:341
12492089912837,75cyclictest961-21snmpd00:16:553
1623991270,31rtkit-daemon994-21lldpd19:53:300
1249208991235,45cyclictest1315030-21grep22:15:163
1249208991233,9cyclictest0-21swapper/320:54:593
1249208991233,22cyclictest0-21swapper/323:05:203
1249208991219,17cyclictest994-21lldpd19:31:363
1249208991208,9cyclictest141rcu_preempt21:18:583
1249202991193,25cyclictest961-21snmpd21:34:591
1249208991173,9cyclictest0-21swapper/319:19:593
12492089911211,14cyclictest961-21snmpd00:27:393
1249208991112,16cyclictest0-21swapper/322:42:033
12491989910949,51cyclictest881337-21WRRende~ckend#120:05:380
12491989910912,56cyclictest881253-21Renderer00:16:550
1249208991085,33cyclictest994-21lldpd23:58:393
1249202991084,98cyclictest880974-21IPDL14:31:441
1249208991075,9cyclictest141rcu_preempt23:37:373
1249208991074,16cyclictest994-21lldpd22:30:173
12492089910711,11cyclictest961-21snmpd23:50:083
1249208991063,59cyclictest994-21lldpd20:49:563
1249208991063,16cyclictest994-21lldpd23:49:533
12492089910519,9cyclictest994-21lldpd19:59:523
1249202991054,95cyclictest994-21lldpd22:57:191
1249202991053,89cyclictest0-21swapper/119:19:591
128817321040,8sleep2994-21lldpd21:00:002
1249208991047,92cyclictest881369-21Timer20:18:243
1249208991043,13cyclictest994-21lldpd22:12:333
1249208991043,12cyclictest994-21lldpd23:31:593
1249202991043,56cyclictest880898-21Xwayland19:12:191
12492029910411,8cyclictest231irq_work/121:55:301
1249208991034,91cyclictest1753-21gnome-shell19:46:593
12492029910343,20cyclictest881356-21Isolated14:31:441
1249202991032,14cyclictest994-21lldpd21:50:151
129183021020,4sleep20-21swapper/221:10:022
1249208991022,84cyclictest0-21swapper/320:42:403
12492029910244,55cyclictest880898-21Xwayland22:45:491
1249202991023,95cyclictest880898-21Xwayland22:30:171
1249202991023,12cyclictest1753-21gnome-shell23:45:081
1249202991022,95cyclictest1753-21gnome-shell21:09:271
1249202991022,95cyclictest1753-21gnome-shell21:04:121
1249202991022,13cyclictest961-21snmpd19:43:061
1249202991022,13cyclictest961-21snmpd19:43:061
12492029910219,15cyclictest881369-21Timer23:39:211
12492029910213,71cyclictest994-21lldpd23:03:501
1249208991014,14cyclictest961-21snmpd20:32:253
1249208991013,50cyclictest994-21lldpd19:29:503
1249208991013,14cyclictest994-21lldpd22:06:163
1249208991013,11cyclictest994-21lldpd22:20:473
1249205991013,52cyclictest881356-21Isolated14:31:442
1249202991013,10cyclictest0-21swapper/123:32:221
1249202991012,95cyclictest1753-21gnome-shell21:42:141
1249208991003,13cyclictest961-21snmpd00:39:413
1249208991003,12cyclictest994-21lldpd23:22:583
1249208991003,11cyclictest994-21lldpd00:11:393
1249208991002,17cyclictest1220806-21kworker/3:022:25:593
1249208991002,11cyclictest994-21lldpd21:53:003
12492029910042,18cyclictest994-21lldpd22:23:171
1249202991003,87cyclictest881253-21Renderer22:07:151
12491989910062,14cyclictest13-21ksoftirqd/023:30:220
1249208999935,25cyclictest994-21lldpd21:22:283
124920899992,13cyclictest961-21snmpd23:11:353
124920299995,89cyclictest26-21ksoftirqd/120:41:551
124920299992,93cyclictest961-21snmpd21:26:281
124920299992,93cyclictest961-21snmpd21:26:271
124920899984,15cyclictest881356-21Isolated14:31:443
124920899983,89cyclictest994-21lldpd20:35:403
124920899983,10cyclictest994-21lldpd00:33:113
124920899982,13cyclictest880898-21Xwayland00:22:253
124920599983,22cyclictest141rcu_preempt23:51:232
1249202999841,53cyclictest880970-21Timer19:49:371
124920299983,11cyclictest994-21lldpd20:23:391
124920899972,9cyclictest994-21lldpd21:27:593
124920899972,9cyclictest994-21lldpd21:27:593
124920899972,11cyclictest994-21lldpd19:38:523
124920599974,48cyclictest994-21lldpd00:36:552
1249202999780,12cyclictest994-21lldpd20:33:401
124920299973,91cyclictest0-21swapper/119:31:361
124920299973,89cyclictest0-21swapper/120:17:091
124920299972,91cyclictest880898-21Xwayland19:37:511
124920899969,84cyclictest42-21ksoftirqd/322:58:333
124920899968,56cyclictest141rcu_preempt00:07:093
124920899963,87cyclictest994-21lldpd21:10:123
124920899962,90cyclictest880898-21Xwayland21:57:153
124920299967,85cyclictest26-21ksoftirqd/100:38:261
1249202999639,16cyclictest994-21lldpd22:52:341
124920299962,85cyclictest881253-21Renderer23:28:511
124920299962,52cyclictest1286010-21kworker/u8:320:57:561
124920299962,11cyclictest881251-21Softwar~cThread20:35:251
124920899953,9cyclictest0-21swapper/322:47:343
124920899953,82cyclictest0-21swapper/320:59:423
124920899953,7cyclictest0-21swapper/321:41:293
124920299954,87cyclictest26-21ksoftirqd/100:05:591
124920299953,88cyclictest0-21swapper/100:27:111
124920299952,89cyclictest0-21swapper/123:50:081
124919899952,90cyclictest880898-21Xwayland20:28:540
124919899952,90cyclictest880898-21Xwayland20:28:540
124920899943,87cyclictest0-21swapper/321:01:273
124920899943,18cyclictest994-21lldpd23:28:513
124920899942,88cyclictest961-21snmpd20:26:543
124920899942,88cyclictest961-21snmpd20:26:543
124920899942,10cyclictest1353371-21chrt00:04:583
124920299943,8cyclictest0-21swapper/100:03:541
124920299942,86cyclictest994-21lldpd19:54:221
12958762930,4chrt1295877-21/usr/sbin/munin21:20:252
124920899932,87cyclictest42-21ksoftirqd/319:12:493
124920899932,7cyclictest0-21swapper/322:52:493
124920599936,80cyclictest881337-21WRRende~ckend#121:51:302
124920299933,86cyclictest0-21swapper/120:52:411
124920299933,7cyclictest231irq_work/100:33:261
124920299933,26cyclictest141rcu_preempt23:41:521
124920299933,26cyclictest141rcu_preempt23:41:521
124920299932,87cyclictest231irq_work/121:35:291
124920299932,87cyclictest1260268-21kworker/1:119:57:591
124920299932,86cyclictest994-21lldpd20:14:381
1249202999313,39cyclictest880965-21glean.dispatche21:22:581
12956672920,4sleep00-21swapper/021:20:190
124920899923,8cyclictest0-21swapper/319:54:593
124920899922,85cyclictest961-21snmpd19:40:363
124920899922,85cyclictest961-21snmpd19:40:363
124920299923,85cyclictest0-21swapper/123:14:511
124920299922,86cyclictest880898-21Xwayland21:18:431
124920299922,7cyclictest0-21swapper/122:41:331
1249202999210,27cyclictest881253-21Renderer20:45:251
124920899914,13cyclictest881356-21Isolated14:31:443
124920899913,8cyclictest0-21swapper/320:05:533
124920899913,17cyclictest881364-21IPC3
124920899913,17cyclictest881364-21IPC3
1249202999128,8cyclictest994-21lldpd21:49:301
124920299912,86cyclictest1260268-21kworker/1:122:13:591
13409242900,5sleep2331rcuc/223:28:212
13092772900,5sleep20-21swapper/221:57:162
1249208999011,22cyclictest0-21swapper/321:31:433
1249202999032,8cyclictest880898-21Xwayland00:22:401
124920899893,10cyclictest961-21snmpd20:04:083
124894628972,9sleep10-21swapper/119:08:171
124920299887,8cyclictest0-21swapper/119:26:501
1249202998779,5cyclictest0-21swapper/123:05:591
1249202998779,5cyclictest0-21swapper/120:26:541
1249202998779,5cyclictest0-21swapper/120:26:541
1249198998735,47cyclictest881337-21WRRende~ckend#119:25:500
1249205998628,48cyclictest881337-21WRRende~ckend#120:09:532
124920599853,78cyclictest0-21swapper/223:17:212
124920599852,50cyclictest880898-21Xwayland22:53:342
124920299852,9cyclictest0-21swapper/120:08:531
124920899842,58cyclictest880898-21Xwayland21:06:573
1249208998417,22cyclictest994-21lldpd19:20:503
1249205998474,6cyclictest1259168-21taskset19:35:212
124920299843,11cyclictest961-21snmpd23:17:361
1249202998419,62cyclictest961-21snmpd19:20:351
1249198998470,10cyclictest1252105-21if_enp2s019:15:200
1249205998324,50cyclictest881337-21WRRende~ckend#120:22:242
1249202998375,5cyclictest0-21swapper/100:15:391
124920299833,75cyclictest881337-21WRRende~ckend#122:15:161
1249205998222,16cyclictest881253-21Renderer21:28:432
1249205998222,16cyclictest881253-21Renderer21:28:432
1249202998275,5cyclictest0-21swapper/123:58:091
124920299812,75cyclictest1314792-21kworker/1:222:27:011
1249198998169,7cyclictest994-21lldpd19:55:220
1249205998056,13cyclictest34-21ksoftirqd/223:32:522
1249198998070,6cyclictest961-21snmpd23:37:590
1249198998022,44cyclictest881253-21Renderer20:36:550
124893028052,17sleep00-21swapper/019:08:040
124876728055,15sleep20-21swapper/219:06:212
124920599794,47cyclictest961-21snmpd21:45:442
124920899788,22cyclictest994-21lldpd21:46:153
1249205997866,7cyclictest994-21lldpd00:20:252
124919899785,11cyclictest961-21snmpd20:53:560
124920899772,45cyclictest994-21lldpd23:18:363
124920299773,11cyclictest994-21lldpd22:04:161
1249198997756,13cyclictest881253-21Renderer19:20:350
124919899773,71cyclictest0-21swapper/022:48:340
1249198997729,8cyclictest0-21swapper/000:10:090
1249198997722,9cyclictest13-21ksoftirqd/021:25:280
1249198997722,9cyclictest13-21ksoftirqd/021:25:280
124920599763,52cyclictest880898-21Xwayland00:05:392
124919899769,21cyclictest141rcu_preempt22:43:180
1249198997657,12cyclictest881253-21Renderer22:14:150
1249198997610,53cyclictest961-21snmpd00:32:560
162399750,6rtkit-daemon0-21swapper/120:00:071
124920899751,43cyclictest994-21lldpd21:39:443
1249198997566,6cyclictest0-21swapper/020:24:590
124919899752,64cyclictest881253-21Renderer22:59:340
124875227552,15sleep30-21swapper/319:06:183
162399740,48rtkit-daemon961-21snmpd22:38:092
13034282740,17sleep234-21ksoftirqd/221:40:382
124919899743,63cyclictest1753-21gnome-shell21:09:590
124919899743,25cyclictest881253-21Renderer22:17:020
124919899743,25cyclictest880898-21Xwayland20:48:400
124919899742,26cyclictest881253-21Renderer21:13:420
1249198997415,52cyclictest994-21lldpd20:33:590
1249205997352,6cyclictest34-21ksoftirqd/223:13:052
1249205997352,6cyclictest34-21ksoftirqd/221:01:122
124920599731,10cyclictest880898-21Xwayland20:32:242
1249198997322,7cyclictest0-21swapper/019:30:500
124920599728,57cyclictest880898-21Xwayland23:46:522
124920599724,15cyclictest881335-21WRScene~ilder#120:12:332
1249198997229,13cyclictest13-21ksoftirqd/023:28:510
124919899722,62cyclictest881253-21Renderer23:42:220
124919899722,62cyclictest881253-21Renderer23:42:210
124919899722,27cyclictest994-21lldpd20:59:120
124920599713,61cyclictest880898-21Xwayland19:29:352
124920599713,59cyclictest881253-21Renderer21:10:422
124920599713,25cyclictest880898-21Xwayland19:16:342
1249205997123,42cyclictest34-21ksoftirqd/223:24:212
1249205997114,51cyclictest994-21lldpd23:09:352
124919899712,62cyclictest880898-21Xwayland19:49:370
1249198997112,16cyclictest994-21lldpd00:05:390
124920899702,8cyclictest141rcu_preempt22:39:033
124920899702,10cyclictest994-21lldpd22:05:013
124920599702,60cyclictest881253-21Renderer00:02:392
1249198997058,7cyclictest994-21lldpd19:11:590
124919899705,57cyclictest881253-21Renderer00:20:550
124919899702,60cyclictest881253-21Renderer22:38:330
124920599693,58cyclictest880898-21Xwayland23:39:222
124920599692,62cyclictest881253-21Renderer20:19:092
124920299692,6cyclictest0-21swapper/122:35:481
124919899698,53cyclictest881253-21Renderer21:19:580
1249198996956,8cyclictest881337-21WRRende~ckend#121:32:440
1249198996936,24cyclictest881253-21Renderer23:58:390
1249198996924,33cyclictest881356-21Isolated14:31:440
1249205996860,5cyclictest0-21swapper/219:55:222
124920599682,60cyclictest881253-21Renderer21:31:142
124920599682,56cyclictest881253-21Renderer22:01:302
124920599681,6cyclictest141rcu_preempt22:22:012
124919899684,19cyclictest880898-21Xwayland22:00:160
1249205996757,6cyclictest0-21swapper/200:27:112
1249205996747,13cyclictest994-21lldpd00:17:402
124920599673,18cyclictest881337-21WRRende~ckend#123:41:522
124920599673,18cyclictest881337-21WRRende~ckend#123:41:522
124920599672,58cyclictest880898-21Xwayland22:40:472
1249205996719,8cyclictest881337-21WRRende~ckend#121:36:142
1249205996712,44cyclictest0-21swapper/222:59:592
1249198996761,4cyclictest0-21swapper/023:24:590
1249198996748,13cyclictest881364-21IPC0
1249198996748,12cyclictest880898-21Xwayland22:05:310
124919899674,16cyclictest881356-21Isolated14:31:440
1249198996714,49cyclictest880974-21IPDL14:31:440
1249208996618,11cyclictest1275102-21ntp_kernel_err20:20:283
1249205996653,8cyclictest994-21lldpd22:10:312
1249198996655,7cyclictest961-21snmpd23:52:380
1249198996649,12cyclictest881364-21IPC0
124919899664,15cyclictest881356-21Isolated14:31:440
124919899663,16cyclictest880898-21Xwayland23:10:500
1249205996548,12cyclictest994-21lldpd19:48:592
124920599654,14cyclictest961-21snmpd00:33:262
1249205996541,16cyclictest881369-21Timer20:29:242
1249205996541,16cyclictest881369-21Timer20:29:242
124920599653,18cyclictest880898-21Xwayland20:39:402
1249205996516,44cyclictest0-21swapper/223:01:502
1249198996557,5cyclictest0-21swapper/022:27:590
124919899654,53cyclictest881356-21Isolated14:31:440
1249198996540,10cyclictest13-21ksoftirqd/021:45:440
124919899653,15cyclictest994-21lldpd22:20:170
124919899652,16cyclictest880898-21Xwayland19:38:360
1249198996511,11cyclictest881337-21WRRende~ckend#121:02:120
124920599644,12cyclictest0-21swapper/223:57:382
124920599643,56cyclictest0-21swapper/222:10:012
1249205996420,11cyclictest141rcu_preempt22:15:592
1249198996448,12cyclictest961-21snmpd22:54:340
1249198996446,12cyclictest13-21ksoftirqd/023:00:500
124919899643,18cyclictest881337-21WRRende~ckend#120:12:080
162399630,49rtkit-daemon961-21snmpd19:43:510
162399630,49rtkit-daemon961-21snmpd19:43:510
124920599634,52cyclictest1753-21gnome-shell19:14:592
124920599634,14cyclictest994-21lldpd19:50:222
124919899634,51cyclictest961-21snmpd20:00:220
1249198996314,45cyclictest961-21snmpd00:29:250
1249205996252,6cyclictest0-21swapper/222:49:342
124919899622,18cyclictest881253-21Renderer23:46:380
124920599612,16cyclictest881253-21Renderer20:41:102
1249205996117,11cyclictest1261787-21Classif~14:31:442
1249205996117,11cyclictest1261787-21Classif~14:31:442
124919899613,49cyclictest994-21lldpd20:15:090
124920599603,50cyclictest881337-21WRRende~ckend#100:11:542
124920599593,49cyclictest881356-21Isolated14:31:442
124920599593,48cyclictest994-21lldpd22:27:332
124920599592,52cyclictest881337-21WRRende~ckend#119:22:352
124919899599,9cyclictest0-21swapper/021:56:300
124920599584,11cyclictest0-21swapper/220:45:102
124920599582,51cyclictest1316279-21kworker/u8:022:33:592
124919899583,11cyclictest994-21lldpd23:07:050
124920599573,12cyclictest881251-21Softwar~cThread20:54:412
124919899573,11cyclictest881337-21WRRende~ckend#120:40:400
124920599543,10cyclictest961-21snmpd19:30:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional