You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 07:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Sun Mar 29, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12878211996,17sleep30-21swapper/319:07:273
13065211794,18sleep00-21swapper/019:09:520
11223211693,16sleep20-21swapper/219:05:022
13031210695,6sleep10-21swapper/119:09:251
107872680,0sleep20-21swapper/222:30:172
107872680,0sleep20-21swapper/222:30:162
13106993918,19cyclictest0-21swapper/319:45:123
178072300,1sleep229-21rcuc/221:30:322
104362250,0sleep30-21swapper/322:30:143
104362250,0sleep30-21swapper/322:30:133
1310399240,2cyclictest0-21swapper/022:20:030
1310599230,22cyclictest0-21swapper/222:20:012
1310699210,19cyclictest0-21swapper/321:50:103
1310499210,19cyclictest0-21swapper/100:05:111
1310699190,2cyclictest0-21swapper/319:10:103
1310499190,1cyclictest0-21swapper/123:35:121
171042180,1sleep117105-21sh23:44:511
13106991813,3cyclictest7904-21ssh23:35:123
1310599180,2cyclictest0-21swapper/219:15:112
1310499180,4cyclictest0-21swapper/123:25:051
1310499180,16cyclictest0-21swapper/121:50:111
121782170,2sleep212183-21ntp_kernel_pll_20:25:172
1310699160,14cyclictest0-21swapper/321:32:083
13105991612,2cyclictest29-21rcuc/219:20:132
13104991613,2cyclictest29177-21kworker/u8:223:02:051
1310499161,13cyclictest23292-21munin-node1
1310499160,14cyclictest0-21swapper/121:35:121
1310699151,12cyclictest0-21swapper/323:48:353
1310699150,13cyclictest0-21swapper/323:41:483
1310599151,12cyclictest0-21swapper/221:00:132
1310599151,12cyclictest0-21swapper/200:14:222
13105991510,3cyclictest7897-21ntp_states21:20:182
13105991510,3cyclictest19823-21ssh22:06:022
1310599150,13cyclictest0-21swapper/200:35:142
1310599150,13cyclictest0-21swapper/200:00:542
1310599150,12cyclictest32200-21latency_hist22:20:012
13104991512,2cyclictest31420-21kworker/u8:120:25:051
13104991512,2cyclictest18949-21kworker/u8:021:30:041
13104991512,2cyclictest0-21swapper/119:21:021
1310499151,12cyclictest28875-21users19:46:031
13104991510,3cyclictest6985-21apt-get20:15:051
1310499150,13cyclictest0-21swapper/123:11:171
1310499150,13cyclictest0-21swapper/122:39:281
1310399151,12cyclictest7958-21ssh21:20:180
13103991510,3cyclictest5266-21irqstats00:05:160
1310699149,3cyclictest29715-21munin-node3
1310699149,3cyclictest0-21swapper/319:30:123
13106991411,2cyclictest29672-21ssh23:24:093
13106991411,2cyclictest0-21swapper/323:50:283
13106991411,2cyclictest0-21swapper/322:52:183
13106991411,2cyclictest0-21swapper/322:45:293
13106991411,2cyclictest0-21swapper/300:02:093
1310699140,2cyclictest0-21swapper/321:47:293
1310699140,2cyclictest0-21swapper/321:43:283
1310699140,13cyclictest0-21swapper/323:15:083
1310699140,13cyclictest0-21swapper/322:02:593
1310699140,12cyclictest0-21swapper/323:09:183
1310699140,12cyclictest0-21swapper/322:05:143
1310599149,3cyclictest0-21swapper/223:45:042
1310599142,6cyclictest0-21swapper/222:45:002
13105991410,2cyclictest301ktimersoftd/219:36:022
1310599140,12cyclictest31-21ksoftirqd/223:55:352
1310599140,12cyclictest24114-21ssh21:37:132
1310599140,12cyclictest0-21swapper/223:02:462
1310499142,6cyclictest0-21swapper/122:45:001
13104991411,2cyclictest0-21swapper/121:42:231
13104991411,2cyclictest0-21swapper/119:30:121
1310499140,3cyclictest16045-21ls19:15:161
1310499140,2cyclictest0-21swapper/120:00:201
1310499140,13cyclictest0-21swapper/122:16:131
1310499140,12cyclictest0-21swapper/100:15:391
13103991411,2cyclictest29232-21diskmemload21:49:290
13103991411,2cyclictest0-21swapper/023:38:200
13103991411,2cyclictest0-21swapper/023:03:300
13103991411,2cyclictest0-21swapper/000:02:000
1310399140,2cyclictest0-21swapper/023:48:190
1310399140,2cyclictest0-21swapper/022:26:300
1310399140,2cyclictest0-21swapper/020:59:590
1310399140,13cyclictest0-21swapper/021:25:100
1310399140,12cyclictest20180-21ssh22:06:180
1310399140,12cyclictest0-21swapper/022:42:250
1310399140,12cyclictest0-21swapper/022:13:150
1310399140,12cyclictest0-21swapper/022:00:180
1310399140,12cyclictest0-21swapper/019:10:120
12982140,2sleep129232-21diskmemload22:20:301
1310699132,6cyclictest0-21swapper/321:00:003
13106991310,2cyclictest0-21swapper/323:12:093
13106991310,2cyclictest0-21swapper/321:59:183
13106991310,2cyclictest0-21swapper/321:16:013
13106991310,2cyclictest0-21swapper/300:23:393
13106991310,2cyclictest0-21swapper/300:16:583
13106991310,2cyclictest0-21swapper/300:07:503
1310699130,2cyclictest6947-21ssh21:20:123
1310699130,2cyclictest0-21swapper/323:57:403
1310699130,2cyclictest0-21swapper/322:21:383
1310699130,12cyclictest0-21swapper/322:59:193
1310699130,12cyclictest0-21swapper/322:25:293
1310699130,12cyclictest0-21swapper/321:10:203
1310699130,12cyclictest0-21swapper/300:14:093
1310599138,3cyclictest29997-21ssh00:30:422
1310599133,6cyclictest15599-21ssh22:01:402
1310599131,6cyclictest0-21swapper/221:55:002
1310599131,2cyclictest31-21ksoftirqd/222:49:462
13105991310,2cyclictest0-21swapper/223:40:212
13105991310,2cyclictest0-21swapper/223:39:142
13105991310,2cyclictest0-21swapper/223:23:042
13105991310,2cyclictest0-21swapper/222:56:062
13105991310,2cyclictest0-21swapper/222:13:542
13105991310,2cyclictest0-21swapper/221:40:352
13105991310,2cyclictest0-21swapper/221:25:362
13105991310,2cyclictest0-21swapper/219:30:172
1310599130,2cyclictest31-21ksoftirqd/223:50:112
1310599130,2cyclictest0-21swapper/223:06:122
1310599130,12cyclictest0-21swapper/223:10:152
1310499131,9cyclictest1992-21ssh22:55:141
1310499131,10cyclictest815-21wc00:35:001
13104991310,2cyclictest9122-21kworker/u8:123:17:041
13104991310,2cyclictest29232-21diskmemload23:30:071
13104991310,2cyclictest29232-21diskmemload21:56:321
13104991310,2cyclictest0-21swapper/123:05:061
13104991310,2cyclictest0-21swapper/122:50:171
13104991310,2cyclictest0-21swapper/122:30:071
13104991310,2cyclictest0-21swapper/122:30:071
13104991310,2cyclictest0-21swapper/122:01:071
13104991310,2cyclictest0-21swapper/121:29:571
13104991310,2cyclictest0-21swapper/100:36:381
1310499130,2cyclictest29232-21diskmemload21:24:471
1310499130,2cyclictest29232-21diskmemload00:23:471
1310499130,2cyclictest0-21swapper/122:07:071
1310499130,2cyclictest0-21swapper/119:55:521
1310499130,2cyclictest0-21swapper/119:10:111
1310499130,12cyclictest0-21swapper/123:45:181
1310499130,12cyclictest0-21swapper/122:26:011
1310499130,11cyclictest0-21swapper/120:31:011
1310399138,3cyclictest31109-21rm22:51:040
13103991310,2cyclictest0-21swapper/023:25:500
13103991310,2cyclictest0-21swapper/022:37:070
13103991310,2cyclictest0-21swapper/021:55:120
13103991310,2cyclictest0-21swapper/021:44:490
1310399130,2cyclictest0-21swapper/023:11:500
1310399130,2cyclictest0-21swapper/022:48:200
1310399130,2cyclictest0-21swapper/022:30:170
1310399130,2cyclictest0-21swapper/022:30:160
1310399130,2cyclictest0-21swapper/000:20:110
1310399130,12cyclictest0-21swapper/023:34:190
1310399130,12cyclictest0-21swapper/023:20:000
1310399130,12cyclictest0-21swapper/021:35:090
1310399130,12cyclictest0-21swapper/000:31:400
1310399130,12cyclictest0-21swapper/000:15:110
1310399130,11cyclictest0-21swapper/021:32:020
135042120,2sleep013506-21ssh23:06:270
1310699129,2cyclictest7787-21ssh23:00:393
1310699129,2cyclictest0-21swapper/321:39:003
1310699129,2cyclictest0-21swapper/300:25:103
1310699127,3cyclictest0-21swapper/322:42:183
1310699122,6cyclictest0-21swapper/321:09:593
1310699122,5cyclictest0-21swapper/319:25:013
1310699120,2cyclictest0-21swapper/320:25:123
1310699120,11cyclictest0-21swapper/323:30:533
1310699120,11cyclictest0-21swapper/323:26:583
1310699120,11cyclictest0-21swapper/322:37:393
1310699120,11cyclictest0-21swapper/322:19:493
1310699120,11cyclictest0-21swapper/300:35:153
1310599129,2cyclictest0-21swapper/223:30:022
1310599129,2cyclictest0-21swapper/223:28:032
1310599129,2cyclictest0-21swapper/221:47:462
1310599129,2cyclictest0-21swapper/200:26:552
1310599129,2cyclictest0-21swapper/200:20:222
1310599122,7cyclictest0-21swapper/221:06:582
1310599121,7cyclictest29232-21diskmemload00:06:012
1310599120,2cyclictest0-21swapper/222:51:022
1310599120,2cyclictest0-21swapper/220:21:032
1310599120,2cyclictest0-21swapper/200:16:122
1310599120,11cyclictest30916-21ssh21:10:442
1310599120,11cyclictest0-21swapper/223:17:122
1310599120,11cyclictest0-21swapper/222:35:252
1310599120,11cyclictest0-21swapper/222:26:332
1310599120,11cyclictest0-21swapper/221:51:012
1310599120,11cyclictest0-21swapper/221:17:122
1310599120,11cyclictest0-21swapper/219:41:042
1310499129,2cyclictest20586-21switchtime19:26:021
1310499129,2cyclictest0-21swapper/121:11:071
1310499129,2cyclictest0-21swapper/100:25:281
1310499129,2cyclictest0-21swapper/100:10:071
1310499127,3cyclictest24573-21sed20:56:021
1310499124,6cyclictest5273-21aten2.4_r3power20:10:141
1310499122,7cyclictest0-21swapper/120:38:591
1310499121,3cyclictest1916-21munin-node1
13104991210,1cyclictest0-21swapper/100:00:481
1310499120,2cyclictest0-21swapper/123:20:201
1310499120,2cyclictest0-21swapper/120:21:001
1310499120,1cyclictest0-21swapper/123:56:271
1310499120,11cyclictest0-21swapper/123:54:401
1310499120,11cyclictest0-21swapper/122:45:071
1310499120,11cyclictest0-21swapper/121:45:061
1310399129,2cyclictest0-21swapper/023:50:120
1310399129,2cyclictest0-21swapper/023:40:110
1310399129,2cyclictest0-21swapper/020:21:010
1310399129,2cyclictest0-21swapper/020:15:130
1310399129,2cyclictest0-21swapper/019:45:150
1310399128,3cyclictest0-21swapper/023:55:180
1310399128,2cyclictest0-21swapper/021:15:200
1310399122,7cyclictest0-21swapper/020:41:580
1310399122,6cyclictest0-21swapper/019:25:010
1310399120,3cyclictest28346-21kworker/u8:223:24:080
1310399120,2cyclictest0-21swapper/021:05:120
1310399120,2cyclictest0-21swapper/000:35:090
1310399120,2cyclictest0-21swapper/000:10:010
1310399120,11cyclictest0-21swapper/022:58:000
1310399120,11cyclictest0-21swapper/021:50:400
1310399120,11cyclictest0-21swapper/021:10:490
1310399120,11cyclictest0-21swapper/019:55:200
1310399120,11cyclictest0-21swapper/000:26:420
1310399120,10cyclictest19524-21kworker/u8:119:36:020
1310699118,2cyclictest0-21swapper/322:14:093
1310699116,3cyclictest20168-21wc20:45:183
1310699112,6cyclictest0-21swapper/320:00:013
1310699111,6cyclictest0-21swapper/320:44:593
1310699111,6cyclictest0-21swapper/320:06:013
1310599118,2cyclictest0-21swapper/219:11:002
1310599116,3cyclictest15965-21ethtool20:35:152
1310599116,3cyclictest0-21swapper/220:15:142
1310599112,6cyclictest0-21swapper/221:00:002
1310599112,5cyclictest0-21swapper/220:41:012
1310599110,9cyclictest3649-21snmpd20:51:252
1310499118,2cyclictest0-21swapper/120:40:121
1310499111,8cyclictest0-21swapper/119:40:121
1310499111,3cyclictest0-21swapper/120:50:161
1310499110,9cyclictest0-21swapper/121:05:111
1310499110,9cyclictest0-21swapper/120:45:141
1310399118,2cyclictest0-21swapper/019:15:150
1310399117,2cyclictest11823-21perl20:25:140
1310399117,2cyclictest0-21swapper/020:00:130
1310399112,6cyclictest0-21swapper/019:45:000
1310399111,6cyclictest0-21swapper/020:34:590
1310399111,5cyclictest0-21swapper/019:55:000
1310399110,9cyclictest0-21swapper/022:16:100
1310699107,2cyclictest0-21swapper/321:26:013
1310699101,6cyclictest7549-21df_abs20:15:133
1310699101,5cyclictest0-21swapper/319:26:013
1310699100,7cyclictest0-21swapper/319:16:593
1310599102,6cyclictest0-21swapper/220:30:152
1310599100,6cyclictest0-21swapper/220:11:032
1310499102,5cyclictest0-21swapper/119:54:581
1310499100,9cyclictest0-21swapper/119:35:131
1310499100,8cyclictest0-21swapper/120:05:111
1310499100,2cyclictest0-21swapper/121:00:111
1310399107,2cyclictest0-21swapper/020:45:170
1310399107,2cyclictest0-21swapper/020:35:170
1310399107,2cyclictest0-21swapper/019:20:120
1310399105,3cyclictest26836-21head21:01:030
131069995,3cyclictest3703-21sshd21:02:353
131069994,3cyclictest5721-21fschecks_time20:10:163
131069994,3cyclictest30438-21pzem_energy19:50:293
131069994,3cyclictest25101-21chrt19:39:143
131069994,3cyclictest22380-21taskset20:50:283
131069991,5cyclictest1223-21cpuspeed_turbos20:00:133
131069991,3cyclictest0-21swapper/320:20:163
131069990,8cyclictest0-21swapper/320:30:123
131059996,2cyclictest0-21swapper/220:45:122
131059994,3cyclictest3062-21head20:05:122
131059994,3cyclictest29797-21df_abs19:50:132
131059994,3cyclictest29205-21latency_hist19:50:012
131059994,3cyclictest19922-21tune2fs19:25:142
131059994,3cyclictest0-21swapper/220:00:012
131039996,2cyclictest0-21swapper/019:30:100
131039992,6cyclictest0-21swapper/020:05:010
131039990,4cyclictest0-21swapper/020:51:050
131069983,3cyclictest15559-21tr20:35:133
131069981,3cyclictest26229-21sed19:40:173
131059983,3cyclictest1164-21egrep20:00:132
131039983,3cyclictest5486-21cpuspeed_turbos20:10:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional