You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-13 - 08:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Jul 13, 2024 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20376996629,1cyclictest0-21swapper/121:16:221
20376996629,1cyclictest0-21swapper/121:16:221
20384993332,1cyclictest23193-21lspci21:16:233
20384993332,1cyclictest23193-21lspci21:16:223
2038099275,21cyclictest28-21ksoftirqd/221:00:002
20380992724,2cyclictest28-21ksoftirqd/222:55:002
20380992724,2cyclictest28-21ksoftirqd/222:55:002
20376992623,2cyclictest18445-21ssh21:42:311
2037699260,25cyclictest14473-21diskmemload22:00:471
2037199260,2cyclictest25144-21/usr/sbin/munin00:00:580
2037199260,2cyclictest1637-21rm21:57:290
20380992521,3cyclictest0-21swapper/221:27:172
2038099250,24cyclictest0-21swapper/222:29:482
2038099250,24cyclictest0-21swapper/222:02:162
2037699250,24cyclictest14473-21diskmemload23:35:181
2037699250,24cyclictest0-21swapper/100:02:551
20371992522,2cyclictest30398-21rm21:23:080
2037199250,24cyclictest5282-21taskset00:12:510
2037199250,24cyclictest14473-21diskmemload23:42:310
2037199250,1cyclictest27281-21ntpq21:51:050
20384992422,1cyclictest0-21swapper/323:33:403
20384992422,1cyclictest0-21swapper/321:36:153
20384992421,2cyclictest2061-21rm21:26:233
2038499240,24cyclictest0-21swapper/323:10:553
2038499240,24cyclictest0-21swapper/321:51:373
2038499240,23cyclictest5196-21ssh23:40:533
2038499240,23cyclictest14473-21diskmemload21:31:193
20380992422,1cyclictest28-21ksoftirqd/219:40:002
2038099240,2cyclictest5198-21rm23:40:522
2038099240,23cyclictest16308-21ssh23:51:052
2038099240,23cyclictest13451-21grep00:21:062
2038099240,0cyclictest14473-21diskmemload22:20:442
2037699240,3cyclictest2098-21turbostat.cron20:45:001
2037699240,24cyclictest14473-21diskmemload21:55:211
2037699240,23cyclictest26830-21apt-get00:35:541
2037699240,23cyclictest12044-21cron23:14:591
2037699240,0cyclictest14473-21diskmemload00:32:551
20371992423,1cyclictest25827-21diskmemload22:21:100
20371992423,1cyclictest14473-21diskmemload23:03:280
20371992421,2cyclictest0-21swapper/021:44:580
2037199240,24cyclictest0-21swapper/022:06:360
2037199240,23cyclictest7461-21rm23:10:030
2037199240,23cyclictest14473-21diskmemload00:19:380
2037199240,23cyclictest0-21swapper/021:16:310
2037199240,23cyclictest0-21swapper/021:16:310
2037199240,0cyclictest0-21swapper/023:09:300
2037199240,0cyclictest0-21swapper/022:12:230
2037199240,0cyclictest0-21swapper/021:26:140
20384992323,0cyclictest0-21swapper/322:16:343
20384992323,0cyclictest0-21swapper/322:11:073
20384992323,0cyclictest0-21swapper/300:21:253
20384992323,0cyclictest0-21swapper/300:15:243
20384992322,1cyclictest0-21swapper/322:01:093
20384992321,1cyclictest26081-21awk00:35:003
20384992321,1cyclictest0-21swapper/321:11:133
2038499231,1cyclictest30175-21rm21:22:473
2038499230,23cyclictest0-21swapper/323:28:123
2038499230,23cyclictest0-21swapper/321:57:393
2038499230,23cyclictest0-21swapper/300:01:513
2038499230,22cyclictest8729-21rm22:05:023
2038499230,22cyclictest22454-21sed22:51:093
2038499230,22cyclictest22454-21sed22:51:093
2038499230,22cyclictest18723-21cat00:26:143
2038499230,22cyclictest18284-21sh21:42:163
2038499230,0cyclictest0-21swapper/323:38:363
2038499230,0cyclictest0-21swapper/322:44:163
20380992323,0cyclictest14473-21diskmemload23:03:522
20380992323,0cyclictest0-21swapper/223:27:502
20380992323,0cyclictest0-21swapper/223:20:072
20380992323,0cyclictest0-21swapper/222:35:472
20380992321,1cyclictest0-21swapper/200:15:002
20380992319,0cyclictest0-21swapper/221:32:082
2038099230,23cyclictest0-21swapper/223:18:482
2038099230,23cyclictest0-21swapper/222:47:562
2038099230,22cyclictest4874-21apt20:49:592
2038099230,22cyclictest23997-21rm21:47:592
2038099230,22cyclictest1541-21ssh21:57:212
2038099230,22cyclictest15307-21rm21:40:212
2038099230,22cyclictest14473-21diskmemload21:39:472
2038099230,1cyclictest0-21swapper/222:56:052
2038099230,1cyclictest0-21swapper/221:17:322
2038099230,1cyclictest0-21swapper/221:17:312
2038099230,0cyclictest0-21swapper/222:41:412
2038099230,0cyclictest0-21swapper/200:30:452
20376992323,0cyclictest0-21swapper/123:09:431
20376992323,0cyclictest0-21swapper/122:07:191
20376992321,1cyclictest0-21swapper/121:30:361
2037699230,2cyclictest32025-21sh21:25:151
2037699230,22cyclictest348-21ssh00:07:471
2037699230,22cyclictest28346-21apt-get19:25:511
2037699230,22cyclictest26822-21avahi-daemon22:20:001
2037699230,22cyclictest16512-21sh22:11:541
2037699230,22cyclictest16393-21sh23:19:091
2037699230,22cyclictest0-21swapper/122:35:501
2037699230,0cyclictest14473-21diskmemload00:14:271
2037699230,0cyclictest0-21swapper/123:45:241
2037699230,0cyclictest0-21swapper/122:55:361
2037699230,0cyclictest0-21swapper/122:20:261
2037699230,0cyclictest0-21swapper/121:36:521
20371992323,0cyclictest22804-21rm23:25:410
20371992323,0cyclictest14473-21diskmemload23:53:080
20371992323,0cyclictest14473-21diskmemload22:40:240
20371992323,0cyclictest14473-21diskmemload00:37:250
20371992323,0cyclictest0-21swapper/022:56:580
20371992323,0cyclictest0-21swapper/000:32:040
20371992323,0cyclictest0-21swapper/000:25:530
20371992321,1cyclictest0-21swapper/021:30:180
20371992320,2cyclictest61622sleep022:35:500
2037199230,23cyclictest0-21swapper/021:35:140
2037199230,22cyclictest6385-21rm22:01:450
2037199230,22cyclictest25616-21munin-run19:19:590
2037199230,22cyclictest22748-21grep19:14:590
2037199230,22cyclictest0-21swapper/021:10:240
2037199230,22cyclictest0-21swapper/019:20:520
2037199230,0cyclictest0-21swapper/023:15:470
2037199230,0cyclictest0-21swapper/022:26:120
2037199230,0cyclictest0-21swapper/000:08:230
20384992222,0cyclictest0-21swapper/323:03:343
20384992222,0cyclictest0-21swapper/322:35:503
20384992222,0cyclictest0-21swapper/300:06:033
20384992221,1cyclictest14473-21diskmemload23:51:073
20384992221,1cyclictest0-21swapper/322:58:023
20384992220,1cyclictest28879-21apt-get22:25:523
20384992220,1cyclictest19773-21apt-get23:55:553
20384992220,1cyclictest0-21swapper/323:05:553
2038499220,2cyclictest22451-21wc20:20:003
2038499220,21cyclictest0-21swapper/323:46:173
2038499220,21cyclictest0-21swapper/321:10:003
2038499220,1cyclictest24153-21rm21:48:143
2038499220,1cyclictest0-21swapper/319:30:003
2038499220,1cyclictest0-21swapper/300:37:033
2038499220,1cyclictest0-21swapper/300:12:273
2038499220,0cyclictest14473-21diskmemload23:24:273
20380992222,0cyclictest0-21swapper/222:32:112
20380992222,0cyclictest0-21swapper/222:06:232
20380992220,2cyclictest615-21polkitd19:30:012
20380992220,1cyclictest17057-21apt-get00:25:542
20380992219,2cyclictest28-21ksoftirqd/221:05:002
2038099220,22cyclictest0-21swapper/223:10:002
2038099220,22cyclictest0-21swapper/222:17:112
2038099220,22cyclictest0-21swapper/222:11:542
2038099220,22cyclictest0-21swapper/200:39:312
2038099220,21cyclictest32606-21ssh00:07:232
2038099220,21cyclictest26026-21apt-get21:20:022
2038099220,21cyclictest25217-21apt-get20:25:542
2038099220,21cyclictest14473-21diskmemload21:10:242
2038099220,21cyclictest0-21swapper/223:45:372
2038099220,1cyclictest0-21swapper/221:51:222
2038099220,0cyclictest0-21swapper/223:59:192
20376992222,0cyclictest0-21swapper/123:00:041
20376992222,0cyclictest0-21swapper/100:28:501
20376992220,1cyclictest25738-21sh21:50:201
20376992220,1cyclictest24905-21latency_hist20:25:001
20376992220,1cyclictest22958-21apt-get23:25:531
20376992220,1cyclictest0-21swapper/123:55:541
20376992220,1cyclictest0-21swapper/123:22:201
20376992220,1cyclictest0-21swapper/121:20:181
20376992219,2cyclictest20629-21sh21:45:331
2037699220,22cyclictest0-21swapper/123:30:271
2037699220,22cyclictest0-21swapper/121:10:451
2037699220,22cyclictest0-21swapper/100:24:121
2037699220,21cyclictest0-21swapper/122:45:551
2037699220,21cyclictest0-21swapper/119:55:521
2037699220,1cyclictest9452-21ssh00:16:411
2037699220,1cyclictest19989-21cron20:15:011
2037699220,1cyclictest0-21swapper/122:41:101
2037699220,1cyclictest0-21swapper/121:10:001
2037699220,0cyclictest0-21swapper/123:50:411
2037699220,0cyclictest0-21swapper/120:05:551
20371992222,0cyclictest14473-21diskmemload23:35:240
20371992222,0cyclictest0-21swapper/023:57:440
20371992222,0cyclictest0-21swapper/000:23:370
20371992221,1cyclictest0-21swapper/023:31:240
20371992220,1cyclictest20777-21apt-get22:50:510
20371992220,1cyclictest20777-21apt-get22:50:510
2037199220,22cyclictest14473-21diskmemload22:30:360
2037199220,22cyclictest0-21swapper/023:45:180
2037199220,21cyclictest0-21swapper/022:15:510
20384992120,1cyclictest0-21swapper/319:20:003
20384992119,1cyclictest32124-21sendmail-msp20:40:013
20384992119,1cyclictest27265-21cron20:30:003
2038499210,21cyclictest0-21swapper/323:17:053
2038499210,21cyclictest0-21swapper/322:45:293
2038499210,21cyclictest0-21swapper/322:20:033
2038499210,20cyclictest0-21swapper/319:10:493
2038499210,0cyclictest0-21swapper/321:00:563
2038499210,0cyclictest0-21swapper/320:00:513
20380992121,0cyclictest0-21swapper/223:35:182
20380992121,0cyclictest0-21swapper/220:35:522
20380992121,0cyclictest0-21swapper/220:30:532
20380992121,0cyclictest0-21swapper/219:50:532
20380992121,0cyclictest0-21swapper/200:15:372
20380992120,0cyclictest32434-21apt-get20:40:522
2038099210,21cyclictest27465-21ssh00:01:512
2038099210,21cyclictest0-21swapper/223:32:182
2038099210,0cyclictest0-21swapper/223:10:552
20376992121,0cyclictest0-21swapper/122:27:251
20376992119,1cyclictest20674-21apt-get19:10:501
2037699210,21cyclictest0-21swapper/123:40:281
2037699210,21cyclictest0-21swapper/122:31:331
2037699210,20cyclictest10029-21apt-get21:00:581
2037699210,0cyclictest0-21swapper/119:45:501
20371992120,0cyclictest3544-21apt-get19:40:530
2037199210,20cyclictest0-21swapper/020:35:520
2037199210,0cyclictest0-21swapper/023:20:380
2037199210,0cyclictest0-21swapper/022:45:020
2037199210,0cyclictest0-21swapper/020:50:530
2037199210,0cyclictest0-21swapper/020:30:520
20384992020,0cyclictest0-21swapper/320:10:523
20384992020,0cyclictest0-21swapper/320:05:543
2038499200,20cyclictest0-21swapper/319:45:503
20380992018,1cyclictest28038-21munin-run19:25:002
20380992018,1cyclictest17632-21perf20:10:002
2038099200,20cyclictest0-21swapper/219:30:012
20376992020,0cyclictest0-21swapper/120:55:531
20376992020,0cyclictest0-21swapper/119:30:011
2037699201,17cyclictest0-21swapper/120:45:531
2037699200,0cyclictest27614-21apt-get20:30:521
20371992020,0cyclictest1102-21apt-get19:35:510
20371992020,0cyclictest0-21swapper/020:40:520
2037199200,0cyclictest0-21swapper/019:30:010
20384991919,0cyclictest0-21swapper/319:35:523
20384991918,1cyclictest27614-21apt-get20:30:523
20380991919,0cyclictest22768-21apt-get20:20:542
20380991919,0cyclictest0-21swapper/219:10:492
20376991919,0cyclictest0-21swapper/120:50:521
20376991919,0cyclictest0-21swapper/120:25:531
20376991917,1cyclictest32129-21awk20:40:001
20371991919,0cyclictest0-21swapper/020:20:520
20371991917,1cyclictest0-21swapper/020:25:530
2037199190,18cyclictest0-21swapper/021:05:550
2037199190,0cyclictest0-21swapper/020:15:500
20384991818,0cyclictest0-21swapper/320:55:523
20384991817,0cyclictest0-21swapper/320:20:523
20384991813,3cyclictest0-21swapper/320:45:003
2038499180,1cyclictest8334-21apt-get19:50:523
20380991818,0cyclictest0-21swapper/219:15:012
20380991817,1cyclictest0-21swapper/219:40:542
2038099180,17cyclictest0-21swapper/219:55:512
20376991818,0cyclictest0-21swapper/119:20:511
2037699180,17cyclictest0-21swapper/120:00:521
20371991818,0cyclictest0-21swapper/021:00:560
20371991817,0cyclictest0-21swapper/020:10:520
20371991816,1cyclictest7607-21apt-get20:55:520
20384991717,0cyclictest0-21swapper/320:50:513
20384991717,0cyclictest0-21swapper/319:40:543
20384991717,0cyclictest0-21swapper/319:30:523
20384991715,1cyclictest2814-21apt-get20:45:533
20380991717,0cyclictest5209-21apt-get20:50:522
20380991716,1cyclictest28-21ksoftirqd/220:20:002
2038099170,0cyclictest5934-21apt-get19:45:502
20376991717,0cyclictest0-21swapper/119:40:541
20371991715,1cyclictest5934-21apt-get19:45:490
20384991614,1cyclictest34-21ksoftirqd/319:56:073
20380991615,1cyclictest13150-21apt-get20:00:512
20384991514,1cyclictest0-21swapper/319:21:003
20376991515,0cyclictest0-21swapper/120:15:511
20376991515,0cyclictest0-21swapper/119:35:511
2037699150,11cyclictest23468-21apt-get19:15:511
20371991515,0cyclictest0-21swapper/020:45:530
20371991515,0cyclictest0-21swapper/020:00:510
20380991414,0cyclictest17941-21apt-get20:10:522
2038099140,14cyclictest0-21swapper/221:05:162
2037699140,14cyclictest0-21swapper/119:50:191
20371991414,0cyclictest0-21swapper/019:55:250
2037199140,14cyclictest0-21swapper/020:05:180
2037199140,14cyclictest0-21swapper/019:50:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional