You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-11-28 - 20:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot7s.osadl.org (updated Tue Nov 28, 2023 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
235202570,2sleep0558-21lldpd08:14:540
308792540,2sleep0558-21lldpd09:41:280
2437223428,2sleep10-21swapper/107:08:521
2459799303,26cyclictest534-21snmpd09:24:001
24597992810,6cyclictest224-21systemd-journal11:30:261
2459299281,2cyclictest534-21snmpd10:02:280
2460299275,18cyclictest224-21systemd-journal09:26:103
2459999273,4cyclictest534-21snmpd09:30:382
2459999271,25cyclictest0-21swapper/211:26:362
2459999271,25cyclictest0-21swapper/211:26:362
2459999260,24cyclictest0-21swapper/209:28:052
24597992622,3cyclictest0-21swapper/112:11:481
140352260,2chrt224-21systemd-journal10:13:423
2459999251,22cyclictest17400-21diskmemload11:00:232
2415922520,1sleep20-21swapper/207:06:082
2460299249,8cyclictest224-21systemd-journal08:25:253
2460299240,2cyclictest0-21swapper/311:52:423
2460299240,22cyclictest0-21swapper/307:40:223
24592992417,5cyclictest13-21ksoftirqd/011:11:090
2460299239,7cyclictest224-21systemd-journal12:33:183
2460299230,19cyclictest0-21swapper/311:44:193
2459999232,20cyclictest37850irq/131-snd_hda07:40:222
24599992312,7cyclictest500-21in:imuxsock12:34:352
2459999230,22cyclictest0-21swapper/211:56:542
2459799239,1cyclictest534-21snmpd09:35:211
2459799231,21cyclictest558-21lldpd07:40:221
2459799230,2cyclictest0-21swapper/110:10:251
2460299225,13cyclictest224-21systemd-journal11:45:173
24602992221,1cyclictest534-21snmpd11:33:573
2460299220,5cyclictest0-21swapper/311:03:293
2459999220,1cyclictest0-21swapper/211:05:032
2459799229,9cyclictest500-21in:imuxsock12:39:041
24597992220,1cyclictest558-21lldpd12:15:571
24597992216,4cyclictest15932-21/usr/sbin/munin11:40:241
2459799220,19cyclictest0-21swapper/109:26:091
2459299224,2cyclictest0-21swapper/012:15:080
24592992220,1cyclictest558-21lldpd11:05:030
2459299220,20cyclictest0-21swapper/009:51:130
2460299219,9cyclictest224-21systemd-journal12:24:573
2460299218,8cyclictest224-21systemd-journal12:00:223
2460299210,1cyclictest224-21systemd-journal09:00:163
2460299210,1cyclictest0-21swapper/311:21:333
24599992112,7cyclictest558-21lldpd12:14:202
2459999210,19cyclictest17553-21tr10:15:312
2459799210,20cyclictest0-21swapper/111:23:451
2459299218,2cyclictest141rcu_preempt12:35:060
2459299211,1cyclictest0-21swapper/010:12:590
2460299203,3cyclictest224-21systemd-journal11:09:103
2460299202,17cyclictest13153-21apache_processe09:30:213
24602992016,1cyclictest224-21systemd-journal10:57:023
24602992011,3cyclictest224-21systemd-journal10:35:303
2459999206,8cyclictest502-21rs:main2
2459999200,20cyclictest0-21swapper/210:44:102
2459799209,3cyclictest558-21lldpd09:50:281
2459799208,2cyclictest558-21lldpd12:23:421
2459799201,3cyclictest17400-21diskmemload09:17:161
2459799201,1cyclictest0-21swapper/109:00:161
24597992010,4cyclictest534-21snmpd10:42:241
2459799200,1cyclictest0-21swapper/111:03:261
2459299205,4cyclictest224-21systemd-journal10:40:260
2459299205,3cyclictest558-21lldpd10:48:540
24592992018,1cyclictest558-21lldpd11:50:200
2459299200,15cyclictest0-21swapper/009:15:320
2460299199,1cyclictest224-21systemd-journal11:29:133
2460299199,1cyclictest224-21systemd-journal11:29:123
2460299192,7cyclictest224-21systemd-journal11:16:493
24602991917,1cyclictest2132-21irqstats10:05:323
2460299190,1cyclictest0-21swapper/310:20:583
2460299190,18cyclictest0-21swapper/312:27:493
2459999193,1cyclictest0-21swapper/209:38:192
2459999192,11cyclictest37850irq/131-snd_hda10:00:582
24599991913,5cyclictest500-21in:imuxsock09:54:332
2459999190,1cyclictest0-21swapper/210:59:532
2459999190,19cyclictest0-21swapper/210:07:312
2459999190,18cyclictest0-21swapper/212:20:282
2459799199,3cyclictest534-21snmpd10:34:081
2459799194,9cyclictest534-21snmpd12:25:071
2459799190,11cyclictest534-21snmpd11:26:371
2459799190,11cyclictest534-21snmpd11:26:371
2459299191,3cyclictest0-21swapper/010:07:170
24592991913,3cyclictest224-21systemd-journal10:25:300
2459299191,11cyclictest558-21lldpd11:15:130
2459299190,1cyclictest0-21swapper/010:54:190
2460299187,7cyclictest0-21swapper/312:15:133
2460299186,6cyclictest558-21lldpd09:46:533
2460299183,9cyclictest224-21systemd-journal10:15:153
2460299183,2cyclictest558-21lldpd10:27:073
24602991817,1cyclictest62050irq/128-enp2s0-09:57:533
24602991815,2cyclictest558-21lldpd08:48:353
2460299181,4cyclictest224-21systemd-journal10:45:223
2460299181,1cyclictest9439-21ssh11:14:523
2459799188,6cyclictest500-21in:imuxsock10:59:061
2459799184,9cyclictest481-21ssh10:46:561
24597991813,2cyclictest29212-21sh10:01:501
24597991812,5cyclictest534-21snmpd11:10:241
2459299188,9cyclictest500-21in:imuxsock09:12:100
24592991816,1cyclictest502-21rs:main0
24592991815,2cyclictest13-21ksoftirqd/010:55:390
24592991814,3cyclictest558-21lldpd11:42:570
2459299181,3cyclictest141rcu_preempt12:21:020
24592991811,3cyclictest500-21in:imuxsock11:58:160
24592991811,2cyclictest224-21systemd-journal08:21:110
2459299180,10cyclictest224-21systemd-journal09:35:290
81142170,1chrt0-21swapper/007:40:310
2460299179,2cyclictest500-21in:imuxsock10:50:273
2460299175,1cyclictest224-21systemd-journal12:36:123
24602991712,4cyclictest30729-21ssh10:03:103
24602991711,5cyclictest3635-21ssh09:23:443
24602991711,2cyclictest224-21systemd-journal09:15:193
2459999172,9cyclictest224-21systemd-journal10:38:042
2459999172,2cyclictest0-21swapper/210:26:322
2459999170,2cyclictest17400-21diskmemload10:45:422
2459999170,16cyclictest17400-21diskmemload09:17:462
2459999170,16cyclictest0-21swapper/211:32:232
2459799178,5cyclictest558-21lldpd09:49:201
2459799176,6cyclictest31800-21apt-get10:25:221
2459799174,5cyclictest558-21lldpd11:45:171
2459799174,4cyclictest558-21lldpd11:17:561
2459799172,12cyclictest534-21snmpd12:00:381
2459799171,5cyclictest224-21systemd-journal10:06:411
24597991712,2cyclictest20848-21ssh09:11:371
24597991712,2cyclictest15747-21cat09:05:241
2459799171,1cyclictest534-21snmpd11:57:231
24597991711,4cyclictest8580-21ssh11:35:281
2459799171,12cyclictest17400-21diskmemload12:06:221
2459799170,2cyclictest0-21swapper/110:16:321
2459299174,3cyclictest224-21systemd-journal11:29:330
2459299174,3cyclictest224-21systemd-journal11:29:330
2459299172,8cyclictest224-21systemd-journal07:25:370
2459299172,8cyclictest224-21systemd-journal07:25:370
2459299171,7cyclictest224-21systemd-journal12:31:500
24592991715,1cyclictest558-21lldpd11:21:140
24592991715,1cyclictest534-21snmpd12:26:420
24592991715,1cyclictest0-21swapper/009:33:050
24592991710,6cyclictest534-21snmpd11:34:500
2459299170,13cyclictest224-21systemd-journal08:43:080
2460299164,3cyclictest224-21systemd-journal12:05:253
2460299163,5cyclictest224-21systemd-journal09:36:053
2460299162,3cyclictest224-21systemd-journal10:34:083
24602991611,4cyclictest18870-21cat08:05:003
24602991610,1cyclictest224-21systemd-journal07:23:263
2460299160,8cyclictest224-21systemd-journal10:40:423
2460299160,1cyclictest0-21swapper/309:14:223
2459999163,8cyclictest502-21rs:main2
2459999162,13cyclictest492-21irqbalance11:41:072
2459999162,13cyclictest17400-21diskmemload11:51:402
24599991612,3cyclictest558-21lldpd10:24:202
2459999161,1cyclictest558-21lldpd11:15:252
2459999160,1cyclictest0-21swapper/212:37:362
2459999160,16cyclictest0-21swapper/211:10:032
2459799168,2cyclictest534-21snmpd10:22:191
2459799165,4cyclictest25468-21unixbench_multi08:15:351
2459799162,9cyclictest17400-21diskmemload09:42:311
2459799162,11cyclictest558-21lldpd10:35:231
24597991611,4cyclictest3190-21ssh11:52:531
2459799161,12cyclictest224-21systemd-journal11:07:111
24597991611,2cyclictest16593-21taskset09:32:111
24597991610,3cyclictest534-21snmpd09:57:421
2459799160,1cyclictest0-21swapper/108:47:561
2459299166,5cyclictest224-21systemd-journal09:20:370
2459299165,8cyclictest7024-21ntp_states09:25:330
2459299165,5cyclictest558-21lldpd11:49:140
2459299165,10cyclictest534-21snmpd09:49:090
2459299164,5cyclictest224-21systemd-journal12:10:270
2459299164,3cyclictest141rcu_preempt10:31:520
24592991610,4cyclictest500-21in:imuxsock12:08:280
2459299160,14cyclictest0-21swapper/007:49:260
2460299155,5cyclictest224-21systemd-journal07:10:223
2460299154,2cyclictest224-21systemd-journal11:55:283
2460299152,4cyclictest224-21systemd-journal09:52:573
2460299151,1cyclictest0-21swapper/309:40:403
2460299151,10cyclictest224-21systemd-journal08:16:543
24602991510,4cyclictest28452-21cut08:25:003
24602991510,4cyclictest2462-21cat08:35:353
2460299150,11cyclictest224-21systemd-journal07:33:343
2459999156,8cyclictest558-21lldpd12:01:312
2459999154,1cyclictest534-21snmpd12:18:122
24599991513,1cyclictest0-21swapper/209:21:372
2459999151,13cyclictest502-21rs:main2
2459799154,5cyclictest224-21systemd-journal07:50:261
24597991511,1cyclictest319162sleep112:33:141
24597991510,2cyclictest9998-21ssh10:53:371
2459299159,5cyclictest502-21rs:main0
2459299157,2cyclictest224-21systemd-journal10:39:350
2459299157,2cyclictest224-21systemd-journal08:25:250
2459299151,9cyclictest224-21systemd-journal08:31:130
2459299151,9cyclictest224-21systemd-journal07:10:170
2459299150,3cyclictest0-21swapper/010:23:590
2460299148,3cyclictest224-21systemd-journal07:18:373
2460299147,2cyclictest224-21systemd-journal07:37:093
2460299146,7cyclictest3322-21apache_accesses08:40:213
2460299145,4cyclictest224-21systemd-journal08:12:023
2460299145,1cyclictest534-21snmpd11:37:503
2460299143,7cyclictest224-21systemd-journal09:07:553
24602991412,1cyclictest0-21swapper/312:11:433
2459999149,2cyclictest20971-21ssh09:11:462
2459999148,3cyclictest502-21rs:main2
2459999142,1cyclictest224-21systemd-journal11:35:422
24599991412,1cyclictest0-21swapper/208:26:322
2459999141,12cyclictest534-21snmpd11:24:192
2459999141,12cyclictest19603-21ssh09:55:342
24599991410,3cyclictest534-21snmpd10:50:362
24599991410,3cyclictest30137-21ssh09:40:472
2459999140,8cyclictest224-21systemd-journal08:11:452
2459999140,5cyclictest224-21systemd-journal11:48:362
2459799141,2cyclictest558-21lldpd07:59:091
24597991411,2cyclictest22483-21sed08:10:311
2459299145,3cyclictest224-21systemd-journal09:59:540
24592991411,2cyclictest18023-21grep10:15:360
2460299137,5cyclictest558-21lldpd08:50:543
2460299133,4cyclictest224-21systemd-journal08:05:063
2460299132,6cyclictest224-21systemd-journal07:45:223
2460299131,6cyclictest224-21systemd-journal08:55:323
24602991310,2cyclictest13532-21chrt07:51:413
2460299130,2cyclictest224-21systemd-journal07:59:093
2459999139,3cyclictest16142-21if_enp2s009:05:272
2459999139,3cyclictest10500-21sed10:10:372
2459999138,4cyclictest558-21lldpd08:06:412
2459999138,2cyclictest224-21systemd-journal08:33:212
2459999131,9cyclictest0-21swapper/207:31:112
24599991310,2cyclictest5827-21fgrep07:35:352
24599991310,2cyclictest37850irq/131-snd_hda10:30:202
2459799139,3cyclictest51252sleep108:41:201
2459799132,3cyclictest502-21rs:main1
24597991311,1cyclictest1589-21chrt07:26:541
24597991311,1cyclictest1589-21chrt07:26:541
24597991310,2cyclictest28133-21taskset08:22:451
24597991310,2cyclictest21031-21seq08:08:291
2459299139,3cyclictest10565-21users11:35:430
2459299135,2cyclictest224-21systemd-journal07:22:190
2459299131,6cyclictest224-21systemd-journal08:15:300
24592991310,2cyclictest4761-21tr07:35:270
24592991310,2cyclictest17400-21diskmemload11:01:410
2459299130,11cyclictest10523-21aten2_r7power_c08:55:250
160142131,5sleep2558-21lldpd07:58:072
2460299129,2cyclictest32109-21test07:25:023
2460299129,2cyclictest32109-21test07:25:013
2460299129,2cyclictest31701-21grep08:30:273
2459999129,2cyclictest28498-21fgrep07:15:352
2459999129,2cyclictest1916-21cat08:35:282
2459999129,2cyclictest18347-21grep08:00:372
2459999129,2cyclictest1600-21taskset07:27:032
2459999129,2cyclictest1600-21taskset07:27:032
2459999129,2cyclictest14132-21sed09:00:362
2459999128,3cyclictest29817-21cut07:20:232
2459999128,3cyclictest13687-21seq07:53:422
2459999126,2cyclictest224-21systemd-journal08:55:312
2459999125,2cyclictest224-21systemd-journal08:15:402
2459999124,4cyclictest224-21systemd-journal08:45:232
2459999124,2cyclictest224-21systemd-journal08:54:242
2459999122,4cyclictest224-21systemd-journal07:10:092
24599991211,1cyclictest32-21ksoftirqd/208:41:172
2459799129,2cyclictest4613-21aten2_r7power_v07:35:261
2459799129,2cyclictest31938-21wc08:30:301
2459799129,2cyclictest2773-21uniq07:30:251
2459799129,2cyclictest26646-21chrt07:11:591
2459799129,2cyclictest12499-21taskset08:58:451
2459799122,2cyclictest16692-21apt-get08:00:221
2459299129,2cyclictest5940-21tr08:45:240
2459299129,2cyclictest1375-21expr08:35:230
2459299129,2cyclictest10118-21taskset08:53:430
2459299128,3cyclictest27319-21cpu07:15:240
2459299125,3cyclictest502-21rs:main0
2459299124,6cyclictest558-21lldpd08:05:230
2459299121,10cyclictest502-21rs:main0
24592991210,1cyclictest0-21swapper/009:05:140
2459299120,5cyclictest224-21systemd-journal08:02:360
2459299120,10cyclictest0-21swapper/007:30:040
2459999118,2cyclictest27360-21tr08:20:322
2459799119,1cyclictest29329-21grep08:25:311
2459799118,2cyclictest9982-21cut07:45:251
2459799118,2cyclictest8366-21tr08:50:221
2459799116,2cyclictest224-21systemd-journal08:35:251
2459799111,9cyclictest558-21lldpd07:23:261
2459299119,1cyclictest0-21swapper/007:50:010
242342102,4sleep30-21swapper/307:07:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional