You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 08:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 14, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
160721240,1sleep10-21swapper/121:15:121
12482211996,17sleep10-21swapper/119:05:561
124522118101,12sleep30-21swapper/319:05:343
12444211692,18sleep00-21swapper/019:05:270
12765210988,16sleep20-21swapper/219:09:342
39472720,1sleep30-21swapper/322:58:313
209152710,0sleep20-21swapper/200:23:452
184642510,1sleep20-21swapper/222:05:202
1282899290,27cyclictest0-21swapper/100:15:111
1282899280,26cyclictest0-21swapper/100:35:141
303612270,1sleep30-21swapper/300:00:093
93392230,1sleep337-21rcuc/320:20:123
1282799231,17cyclictest0-21swapper/023:46:050
1282799231,17cyclictest0-21swapper/023:46:050
12829992218,2cyclictest0-21swapper/220:25:122
1282999220,2cyclictest0-21swapper/220:55:122
1282999220,2cyclictest0-21swapper/220:55:122
1283099211,18cyclictest0-21swapper/300:35:133
1282999210,19cyclictest0-21swapper/219:55:122
1282899210,19cyclictest0-21swapper/120:45:151
1282799210,16cyclictest0-21swapper/023:19:050
85632200,0sleep10-21swapper/121:21:051
1282899200,2cyclictest0-21swapper/123:25:151
12827992017,2cyclictest19227-21kworker/u8:219:34:040
112522200,1sleep3381ktimersoftd/323:05:223
26012190,0sleep00-21swapper/000:38:260
1282799193,2cyclictest0-21swapper/023:07:050
1282799192,2cyclictest0-21swapper/022:01:050
1282799192,15cyclictest0-21swapper/022:44:010
12827991916,2cyclictest9033-21kworker/u8:121:27:040
12827991915,2cyclictest0-21swapper/000:20:140
1282799190,13cyclictest0-21swapper/021:50:040
1282999180,16cyclictest0-21swapper/222:50:142
1282799183,2cyclictest28949-21diskmemload23:40:050
1282799182,2cyclictest0-21swapper/023:53:040
1282799182,2cyclictest0-21swapper/023:38:040
1282799182,2cyclictest0-21swapper/023:14:030
1282799181,2cyclictest0-21swapper/023:23:040
1282799181,2cyclictest0-21swapper/022:10:030
1282799180,3cyclictest0-21swapper/000:30:050
1282799180,2cyclictest0-21swapper/019:14:040
144262170,1sleep00-21swapper/000:16:050
1282799172,6cyclictest0-21swapper/000:10:020
1282799172,2cyclictest0-21swapper/000:25:050
12827991714,2cyclictest9115-21kworker/u8:020:35:040
12827991714,2cyclictest20024-21kworker/u8:020:50:040
1283099160,3cyclictest2221-21wc21:15:163
1283099160,14cyclictest0-21swapper/319:10:143
12829991612,2cyclictest0-21swapper/220:10:242
1282799161,2cyclictest32369-21ssh22:54:420
1282799160,3cyclictest0-21swapper/021:38:040
1282799160,2cyclictest0-21swapper/020:43:040
1282799160,2cyclictest0-21swapper/020:27:030
12830991510,3cyclictest0-21swapper/323:18:163
12829991512,2cyclictest10073-21mailstats20:20:252
12829991510,3cyclictest9186-21cpu22:30:142
1282899155,7cyclictest5911-21gdm-simple-gree22:46:581
12828991512,2cyclictest3452-21kworker/u8:223:54:051
12828991510,3cyclictest15612-21fschecks_time19:15:161
1282899150,13cyclictest0-21swapper/123:10:241
1282799151,12cyclictest7296-21df20:15:160
1282799151,12cyclictest4626-21munin-run20:10:000
12827991510,3cyclictest0-21swapper/020:10:190
1282799150,3cyclictest0-21swapper/022:22:040
1282799150,2cyclictest0-21swapper/023:29:040
1282799150,2cyclictest0-21swapper/023:00:040
1282799150,2cyclictest0-21swapper/022:28:010
1282799150,2cyclictest0-21swapper/021:40:040
1282799150,2cyclictest0-21swapper/020:20:040
1282799150,2cyclictest0-21swapper/019:51:040
12830991411,2cyclictest0-21swapper/320:15:143
12830991410,2cyclictest0-21swapper/323:50:143
1283099140,6cyclictest28949-21diskmemload21:25:593
12829991410,2cyclictest5193-21ssh21:18:352
1282999140,3cyclictest21521-21cut23:50:182
1282999140,2cyclictest0-21swapper/219:20:152
1282999140,12cyclictest28949-21diskmemload00:15:442
1282999140,12cyclictest28949-21diskmemload00:01:022
1282999140,12cyclictest0-21swapper/223:55:532
1282999140,12cyclictest0-21swapper/223:35:422
1282899142,7cyclictest0-21swapper/123:45:011
1282899142,6cyclictest0-21swapper/100:30:011
1282899141,2cyclictest28949-21diskmemload21:25:071
1282899140,12cyclictest0-21swapper/121:57:571
1282799142,2cyclictest0-21swapper/022:35:040
12827991411,2cyclictest0-21swapper/019:55:120
12827991411,2cyclictest0-21swapper/000:01:010
1282799140,2cyclictest6459-21ssh21:20:020
1282799140,2cyclictest0-21swapper/022:30:050
96262130,0sleep10-21swapper/120:20:151
305032131,9sleep030513-21ssh21:45:100
1283099139,2cyclictest0-21swapper/323:30:183
1283099139,2cyclictest0-21swapper/321:40:163
12830991310,2cyclictest29522-21ssh21:10:143
12830991310,2cyclictest28769-21ssh00:31:043
12830991310,2cyclictest0-21swapper/323:03:453
12830991310,2cyclictest0-21swapper/322:21:263
12830991310,2cyclictest0-21swapper/321:52:453
12830991310,2cyclictest0-21swapper/321:45:033
12830991310,2cyclictest0-21swapper/321:20:033
12830991310,2cyclictest0-21swapper/320:30:153
12830991310,2cyclictest0-21swapper/300:21:363
12830991310,2cyclictest0-21swapper/300:11:343
12830991310,2cyclictest0-21swapper/300:05:153
1283099130,2cyclictest0-21swapper/322:15:163
1283099130,2cyclictest0-21swapper/322:08:533
1283099130,12cyclictest0-21swapper/322:34:243
1283099130,12cyclictest0-21swapper/321:05:233
1283099130,12cyclictest0-21swapper/300:17:543
1283099130,11cyclictest7333-21missed_timers23:35:193
1283099130,11cyclictest16065-21apt-get23:45:133
1283099130,11cyclictest16065-21apt-get23:45:133
1282999132,6cyclictest23101-21copy22:44:592
1282999132,6cyclictest0-21swapper/222:30:002
1282999132,6cyclictest0-21swapper/221:00:012
1282999131,6cyclictest0-21swapper/222:15:002
12829991310,2cyclictest0-21swapper/222:40:202
12829991310,2cyclictest0-21swapper/222:37:342
12829991310,2cyclictest0-21swapper/200:35:432
12829991310,2cyclictest0-21swapper/200:30:132
1282999130,12cyclictest0-21swapper/223:33:542
1282999130,11cyclictest5010-21snmpd23:11:182
1282899134,6cyclictest5911-21gdm-simple-gree00:05:581
1282899132,6cyclictest0-21swapper/122:45:001
1282899132,6cyclictest0-21swapper/120:05:001
1282899131,4cyclictest31288-21sed00:00:171
1282899131,10cyclictest0-21swapper/123:05:151
12828991310,2cyclictest0-21swapper/123:20:231
12828991310,2cyclictest0-21swapper/122:19:171
12828991310,2cyclictest0-21swapper/121:50:201
12828991310,2cyclictest0-21swapper/120:30:221
12828991310,2cyclictest0-21swapper/119:35:181
1282899130,6cyclictest0-21swapper/120:20:001
1282899130,2cyclictest0-21swapper/123:46:471
1282899130,2cyclictest0-21swapper/123:46:461
1282899130,2cyclictest0-21swapper/122:30:081
1282899130,2cyclictest0-21swapper/122:20:581
1282899130,2cyclictest0-21swapper/119:25:181
1282899130,12cyclictest0-21swapper/123:35:271
1282899130,12cyclictest0-21swapper/123:30:221
1282899130,11cyclictest28949-21diskmemload23:55:301
1282899130,11cyclictest20318-21ssh21:34:331
1282899130,11cyclictest0-21swapper/122:57:481
1282899130,11cyclictest0-21swapper/120:05:161
1282799132,6cyclictest0-21swapper/023:30:010
1282799132,6cyclictest0-21swapper/019:25:000
12827991310,2cyclictest29321-21ssh22:16:450
12827991310,2cyclictest28949-21diskmemload21:56:250
12827991310,2cyclictest0-21swapper/022:06:200
12827991310,2cyclictest0-21swapper/020:30:150
1282799130,2cyclictest0-21swapper/021:19:040
1282799130,12cyclictest0-21swapper/021:12:320
1282799130,12cyclictest0-21swapper/021:00:150
62322120,2sleep10-21swapper/122:26:121
1283099129,2cyclictest0-21swapper/322:50:223
1283099129,2cyclictest0-21swapper/322:40:243
1283099129,2cyclictest0-21swapper/322:28:153
1283099129,2cyclictest0-21swapper/320:45:183
1283099129,2cyclictest0-21swapper/320:00:113
1283099129,2cyclictest0-21swapper/300:27:053
1283099121,2cyclictest17664-21hddtemp_smartct20:40:173
1283099120,2cyclictest0-21swapper/323:20:013
1283099120,2cyclictest0-21swapper/321:55:233
1283099120,2cyclictest0-21swapper/319:30:163
1283099120,11cyclictest0-21swapper/323:55:063
1283099120,11cyclictest0-21swapper/323:40:563
1283099120,11cyclictest0-21swapper/323:25:023
1283099120,11cyclictest0-21swapper/322:46:433
1283099120,11cyclictest0-21swapper/322:36:363
1283099120,11cyclictest0-21swapper/322:01:533
1283099120,11cyclictest0-21swapper/321:36:543
1283099120,11cyclictest0-21swapper/321:30:253
1282999129,2cyclictest0-21swapper/223:48:242
1282999129,2cyclictest0-21swapper/223:48:232
1282999129,2cyclictest0-21swapper/223:15:222
1282999129,2cyclictest0-21swapper/221:29:252
1282999120,2cyclictest0-21swapper/223:21:332
1282999120,2cyclictest0-21swapper/223:08:222
1282999120,2cyclictest0-21swapper/222:59:332
1282999120,2cyclictest0-21swapper/221:40:352
1282999120,2cyclictest0-21swapper/221:11:152
1282999120,2cyclictest0-21swapper/200:25:232
1282999120,2cyclictest0-21swapper/200:12:322
1282999120,2cyclictest0-21swapper/200:08:412
1282999120,11cyclictest0-21swapper/222:23:022
1282999120,10cyclictest28949-21diskmemload23:43:322
1282899129,2cyclictest28949-21diskmemload23:00:161
1282899129,2cyclictest0-21swapper/122:11:331
1282899129,2cyclictest0-21swapper/122:00:081
1282899129,2cyclictest0-21swapper/100:21:171
1282899127,3cyclictest0-21swapper/119:10:251
1282899122,7cyclictest5911-21gdm-simple-gree20:10:581
1282899122,7cyclictest0-21swapper/100:10:591
1282899121,9cyclictest28949-21diskmemload22:05:161
1282899121,9cyclictest0-21swapper/122:50:281
12828991210,1cyclictest0-21swapper/123:15:571
1282899120,3cyclictest0-21swapper/121:10:221
1282899120,2cyclictest0-21swapper/121:37:081
1282899120,2cyclictest0-21swapper/120:50:241
1282899120,2cyclictest0-21swapper/100:30:211
1282899120,11cyclictest0-21swapper/121:47:461
1282899120,11cyclictest0-21swapper/121:40:181
1282899120,11cyclictest0-21swapper/119:50:171
1282899120,11cyclictest0-21swapper/119:45:171
1282799129,2cyclictest0-21swapper/022:57:310
1282799129,2cyclictest0-21swapper/019:35:130
1282799120,2cyclictest0-21swapper/021:05:130
1282799120,2cyclictest0-21swapper/019:18:010
1282799120,1cyclictest0-21swapper/023:55:010
1282799120,11cyclictest0-21swapper/022:45:290
1282799120,11cyclictest0-21swapper/021:31:090
1282799120,11cyclictest0-21swapper/019:40:270
1282799120,11cyclictest0-21swapper/000:05:570
1283099111,8cyclictest20926-21munin-run19:30:003
1283099111,7cyclictest5911-21gdm-simple-gree19:35:593
1283099110,10cyclictest0-21swapper/323:13:343
1283099110,10cyclictest0-21swapper/322:10:483
1283099110,10cyclictest0-21swapper/319:45:153
1282999118,2cyclictest0-21swapper/221:36:252
1282999118,2cyclictest0-21swapper/221:32:542
1282999118,2cyclictest0-21swapper/221:21:152
1282999112,6cyclictest0-21swapper/219:52:592
1282999111,6cyclictest0-21swapper/220:40:132
1282999110,10cyclictest0-21swapper/223:00:242
1282999110,10cyclictest0-21swapper/222:00:152
1282999110,10cyclictest0-21swapper/221:52:242
1282999110,10cyclictest0-21swapper/221:47:452
1282999110,10cyclictest0-21swapper/220:15:152
1282899117,2cyclictest0-21swapper/119:30:131
1282899112,7cyclictest0-21swapper/120:41:591
1282899112,5cyclictest0-21swapper/119:22:021
1282899111,8cyclictest25141-21munin-node1
1282899111,3cyclictest15057-21munin-node1
1282899110,3cyclictest0-21swapper/120:25:191
1282899110,10cyclictest28949-21diskmemload22:35:241
1282799119,1cyclictest0-21swapper/020:55:150
1282799119,1cyclictest0-21swapper/020:55:150
1282799112,6cyclictest0-21swapper/020:02:580
1282799110,2cyclictest0-21swapper/019:45:040
99302100,1sleep20-21swapper/221:56:182
1283099107,2cyclictest25851-21iostat19:40:173
1283099107,2cyclictest0-21swapper/320:35:223
1283099107,2cyclictest0-21swapper/320:00:003
1283099105,3cyclictest25377-21tr21:00:163
1283099102,6cyclictest0-21swapper/320:55:243
1283099102,6cyclictest0-21swapper/320:55:243
1283099102,5cyclictest0-21swapper/319:54:593
1283099102,5cyclictest0-21swapper/319:19:593
1282999105,3cyclictest30770-21grep23:25:302
1282999105,3cyclictest2134-21grep20:00:252
1282999104,4cyclictest20919-21sh19:30:002
1282999102,6cyclictest0-21swapper/219:10:242
1282999102,5cyclictest0-21swapper/219:41:592
1282999101,3cyclictest22686-21taskset20:52:252
1282999100,8cyclictest0-21swapper/222:10:302
1282999100,7cyclictest0-21swapper/221:08:592
1282899104,2cyclictest0-21swapper/119:55:041
1282899101,7cyclictest0-21swapper/119:40:161
1282899100,9cyclictest0-21swapper/120:57:171
1282899100,9cyclictest0-21swapper/120:57:161
1282899100,8cyclictest27146-21munin-node1
1282799107,2cyclictest0-21swapper/020:45:210
128309996,2cyclictest0-21swapper/320:26:233
128309994,3cyclictest3206-21cut20:05:173
128299994,3cyclictest22897-21sshd19:34:412
128299994,3cyclictest16007-21wc20:35:252
128299994,3cyclictest14438-21unixbench_singl20:30:292
128299993,4cyclictest278-21jbd2/sda1-819:15:182
128299991,6cyclictest0-21swapper/219:35:172
128299990,6cyclictest0-21swapper/219:46:462
128279994,3cyclictest20257-21switchtime19:25:250
128309985,2cyclictest0-21swapper/320:10:063
128309983,3cyclictest21090-21taskset20:50:033
128309980,7cyclictest0-21swapper/319:20:053
128299984,2cyclictest0-21swapper/220:45:162
128299983,3cyclictest2863-21cut20:05:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional