You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 12:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot6s.osadl.org (updated Fri Jan 02, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1189121360,1sleep011892-21sh23:31:090
687211794,18sleep00-21swapper/019:05:440
856211595,15sleep20-21swapper/219:07:542
9882114103,6sleep10-21swapper/119:09:381
69029382,6sleep30-21swapper/319:05:473
9112790,0sleep00-21swapper/021:27:590
253482670,0sleep20-21swapper/221:19:032
98712660,0sleep10-21swapper/100:05:411
1048993315,16cyclictest0-21swapper/220:10:112
1046993315,17cyclictest0-21swapper/020:10:110
104999290,2cyclictest21322-21ssh00:20:133
104699299,18cyclictest0-21swapper/022:15:120
296412280,1sleep10-21swapper/120:20:001
49442240,0sleep20-21swapper/222:46:372
248202240,1sleep0101ktimersoftd/000:24:000
1048992319,2cyclictest0-21swapper/223:30:122
29702220,0sleep20-21swapper/222:07:322
104999221,16cyclictest0-21swapper/323:01:043
104999220,17cyclictest0-21swapper/322:52:053
104899221,19cyclictest0-21swapper/222:15:122
104899220,20cyclictest0-21swapper/200:35:132
74282210,1sleep10-21swapper/120:45:001
298692210,1sleep10-21swapper/122:38:511
104999211,15cyclictest0-21swapper/321:19:043
104999210,16cyclictest0-21swapper/321:36:053
104899210,19cyclictest0-21swapper/220:00:112
104699211,18cyclictest0-21swapper/019:25:110
104699210,19cyclictest0-21swapper/022:35:140
104999201,15cyclictest0-21swapper/300:32:053
104999200,15cyclictest0-21swapper/322:44:053
104999200,15cyclictest0-21swapper/300:16:053
104899200,5cyclictest0-21swapper/220:28:052
104899200,18cyclictest0-21swapper/220:45:112
104699200,18cyclictest0-21swapper/021:05:120
104699200,18cyclictest0-21swapper/020:00:110
104999190,18cyclictest0-21swapper/320:49:053
1048991916,2cyclictest18919-21kworker/u8:022:26:052
1048991916,2cyclictest18919-21kworker/u8:022:03:042
104699190,2cyclictest0-21swapper/021:00:150
1049991815,2cyclictest4209-21kworker/u8:121:33:043
104999180,2cyclictest0-21swapper/320:28:053
104899182,2cyclictest0-21swapper/221:30:052
1048991815,2cyclictest27391-21kworker/u8:023:53:042
104899180,16cyclictest0-21swapper/220:05:122
104799181,2cyclictest23-21ksoftirqd/123:20:121
104999172,11cyclictest0-21swapper/322:37:053
1049991714,2cyclictest27391-21kworker/u8:023:54:053
1049991714,2cyclictest11267-21kworker/u8:221:41:033
1049991713,2cyclictest1990-21kworker/u8:200:39:043
104999170,2cyclictest0-21swapper/322:45:053
104999170,2cyclictest0-21swapper/321:24:053
1048991714,2cyclictest4209-21kworker/u8:121:46:052
1048991714,2cyclictest30512-21kworker/u8:021:29:042
1048991714,2cyclictest18919-21kworker/u8:022:22:052
104899171,3cyclictest14252-21irqstats19:40:162
104899171,2cyclictest0-21swapper/221:37:052
104899170,5cyclictest0-21swapper/223:10:052
104799179,6cyclictest0-21swapper/123:00:001
104799171,3cyclictest0-21swapper/123:40:161
104799170,2cyclictest0-21swapper/122:30:131
33742160,1sleep33380-21aten2.4_r3power19:15:123
104999162,2cyclictest0-21swapper/322:32:053
1049991613,2cyclictest27242-21kworker/u8:223:16:053
1049991613,2cyclictest20941-21kworker/u8:221:25:053
104999161,2cyclictest0-21swapper/323:38:053
104999161,2cyclictest0-21swapper/323:20:053
104999161,2cyclictest0-21swapper/322:23:053
104999160,2cyclictest0-21swapper/322:01:043
104999160,2cyclictest0-21swapper/321:03:053
104999160,2cyclictest0-21swapper/319:45:053
1048991613,2cyclictest30817-21kworker/u8:222:57:052
1048991613,2cyclictest30817-21kworker/u8:222:54:052
1048991613,2cyclictest27423-21kworker/u8:020:16:052
1048991613,2cyclictest13834-21kworker/u8:200:02:052
1048991613,2cyclictest11595-21kworker/u8:021:04:052
104899161,2cyclictest0-21swapper/223:49:052
104899161,13cyclictest0-21swapper/221:10:152
104899160,11cyclictest0-21swapper/223:06:052
104799161,3cyclictest0-21swapper/121:22:181
104799161,13cyclictest21-21rcuc/121:12:301
104799161,13cyclictest0-21swapper/100:30:151
104699161,13cyclictest7458-21runrttasks21:23:120
104999152,5cyclictest0-21swapper/322:10:053
1049991512,2cyclictest7707-21kworker/u8:123:31:043
1049991512,2cyclictest18831-21kworker/u8:123:41:053
1049991512,2cyclictest15212-21kworker/u8:219:59:053
1049991512,2cyclictest13834-21kworker/u8:200:03:043
104999150,5cyclictest0-21swapper/322:06:053
104999150,2cyclictest0-21swapper/323:10:053
104999150,2cyclictest0-21swapper/319:11:053
1048991512,2cyclictest9698-21kworker/u8:019:31:052
1048991512,2cyclictest30817-21kworker/u8:222:43:052
1048991512,2cyclictest3032-21kworker/u8:119:26:052
1048991512,2cyclictest29748-21kworker/u8:220:24:052
1048991512,2cyclictest27423-21kworker/u8:020:32:052
1048991512,2cyclictest25412-21kworker/u8:121:22:052
1048991512,2cyclictest15212-21kworker/u8:219:57:052
1048991512,2cyclictest13015-21kworker/u8:119:47:052
1048991512,2cyclictest10932-21kworker/u8:122:33:032
104899151,12cyclictest5919-21fschecks_count23:25:192
104899150,2cyclictest31-21ksoftirqd/200:29:042
104899150,2cyclictest0-21swapper/223:42:052
104799151,5cyclictest23-21ksoftirqd/120:01:071
104799151,12cyclictest0-21swapper/120:35:061
1047991511,2cyclictest0-21swapper/100:15:151
1046991510,3cyclictest16826-21cat21:45:280
104699150,13cyclictest5627-21pzem_current22:10:200
104699150,13cyclictest0-21swapper/021:52:370
186702140,11sleep20-21swapper/223:02:262
176582140,2sleep00-21swapper/021:10:080
136652140,3sleep013673-21ssh22:56:180
104999141,2cyclictest0-21swapper/300:09:053
1049991411,2cyclictest27929-21kworker/u8:100:28:053
1049991411,2cyclictest24007-21kworker/u8:121:59:053
1049991411,2cyclictest17390-21diskmemload23:45:163
1049991411,2cyclictest0-21swapper/322:25:133
104999140,2cyclictest0-21swapper/319:40:183
104999140,2cyclictest0-21swapper/319:25:053
1048991411,2cyclictest3032-21kworker/u8:119:24:052
1048991411,2cyclictest3032-21kworker/u8:119:18:052
1048991411,2cyclictest27391-21kworker/u8:023:56:052
1048991411,2cyclictest27367-21kworker/u8:019:12:052
1048991411,2cyclictest15985-21kworker/u8:123:17:052
1048991410,2cyclictest3424-21aten2.4-expect20:35:162
104899140,12cyclictest0-21swapper/221:54:422
104899140,12cyclictest0-21swapper/200:22:382
104899140,12cyclictest0-21swapper/200:17:292
104899140,12cyclictest0-21swapper/200:05:142
104799149,3cyclictest4377-21cut00:00:181
104799149,3cyclictest18449-21ssh23:02:001
104799149,3cyclictest10671-21ssh21:39:431
104799141,2cyclictest17390-21diskmemload21:15:251
1047991411,2cyclictest221ktimersoftd/123:35:181
104799141,12cyclictest221ktimersoftd/100:28:051
1047991411,2cyclictest21-21rcuc/122:12:441
1047991411,2cyclictest0-21swapper/123:10:281
104799140,13cyclictest0-21swapper/122:40:151
104799140,12cyclictest23075-21ssh21:53:431
104799140,12cyclictest17390-21diskmemload22:05:311
104799140,12cyclictest15442-21ssh22:21:231
104799140,12cyclictest0-21swapper/122:53:491
104699144,7cyclictest15227-21ssh22:20:580
1046991411,2cyclictest24759-21ssh23:10:110
104699140,12cyclictest17390-21diskmemload23:05:170
104699140,12cyclictest0-21swapper/000:29:010
1049991310,2cyclictest7707-21kworker/u8:123:27:053
1049991310,2cyclictest3364-21kworker/u8:120:51:053
1049991310,2cyclictest23131-21kworker/u8:120:03:053
1049991310,2cyclictest11595-21kworker/u8:021:07:043
1049991310,2cyclictest0-21swapper/321:50:193
104999130,2cyclictest16631-21kworker/3:023:05:153
104999130,2cyclictest0-21swapper/322:15:053
104999130,2cyclictest0-21swapper/321:11:053
104999130,2cyclictest0-21swapper/320:56:053
104999130,2cyclictest0-21swapper/319:23:053
104999130,12cyclictest0-21swapper/321:45:293
1048991310,2cyclictest3364-21kworker/u8:120:55:052
1048991310,2cyclictest3364-21kworker/u8:120:54:052
1048991310,2cyclictest29748-21kworker/u8:220:42:052
1048991310,2cyclictest12540-21kworker/u8:000:30:052
1048991310,2cyclictest10932-21kworker/u8:122:35:042
1048991310,2cyclictest0-21swapper/222:10:202
1048991310,2cyclictest0-21swapper/200:13:592
104899130,3cyclictest0-21swapper/223:23:292
104899130,2cyclictest0-21swapper/223:39:402
104899130,12cyclictest0-21swapper/221:40:262
104799138,3cyclictest32402-21ssh22:04:571
104799138,3cyclictest17864-21tr19:50:141
1047991310,2cyclictest21-21rcuc/122:15:061
1047991310,2cyclictest0-21swapper/123:51:431
1047991310,2cyclictest0-21swapper/123:45:571
1047991310,2cyclictest0-21swapper/123:25:241
1047991310,2cyclictest0-21swapper/123:07:131
104799130,2cyclictest17390-21diskmemload22:27:531
104799130,2cyclictest17390-21diskmemload00:37:261
104799130,12cyclictest0-21swapper/123:15:201
104799130,12cyclictest0-21swapper/121:45:231
104799130,12cyclictest0-21swapper/121:34:001
104799130,12cyclictest0-21swapper/100:20:281
104799130,11cyclictest128072sleep121:40:391
104799130,11cyclictest0-21swapper/120:20:211
104799130,11cyclictest0-21swapper/119:40:201
104699133,6cyclictest17390-21diskmemload23:25:290
1046991310,2cyclictest8161-21ssh00:05:070
1046991310,2cyclictest7823-21ssh00:04:570
1046991310,2cyclictest0-21swapper/023:03:020
1046991310,2cyclictest0-21swapper/021:18:170
1046991310,2cyclictest0-21swapper/000:35:070
1046991310,2cyclictest0-21swapper/000:13:400
104699130,2cyclictest0-21swapper/022:31:090
104699130,12cyclictest10491-21ssh22:53:270
104699130,12cyclictest0-21swapper/023:50:300
104699130,11cyclictest0-21swapper/023:20:310
104699130,11cyclictest0-21swapper/019:20:210
104699130,11cyclictest0-21swapper/000:15:120
92942120,2sleep00-21swapper/021:37:100
104999120,2cyclictest12543-21ssh00:10:103
104999120,2cyclictest0-21swapper/322:55:193
104999120,2cyclictest0-21swapper/319:30:153
104999120,11cyclictest0-21swapper/323:57:363
104999120,11cyclictest0-21swapper/320:35:193
104899129,2cyclictest0-21swapper/221:07:402
104899127,3cyclictest0-21swapper/219:40:002
104899120,11cyclictest0-21swapper/221:56:342
104899120,11cyclictest0-21swapper/219:50:192
104799129,2cyclictest0-21swapper/119:45:171
104799129,2cyclictest0-21swapper/119:35:191
104799129,2cyclictest0-21swapper/119:15:111
104799127,3cyclictest0-21swapper/120:55:131
104799127,3cyclictest0-21swapper/119:30:181
104799122,6cyclictest0-21swapper/119:15:011
104799121,8cyclictest6875-21runrttasks19:21:021
104799120,2cyclictest0-21swapper/123:30:581
104799120,2cyclictest0-21swapper/121:05:161
104799120,2cyclictest0-21swapper/120:05:111
104799120,11cyclictest0-21swapper/123:55:201
104799120,11cyclictest0-21swapper/122:46:201
104799120,11cyclictest0-21swapper/121:55:111
104799120,11cyclictest0-21swapper/121:25:251
104799120,11cyclictest0-21swapper/121:00:231
104799120,11cyclictest0-21swapper/100:10:111
104699129,2cyclictest0-21swapper/022:43:070
104699129,2cyclictest0-21swapper/022:25:100
104699129,2cyclictest0-21swapper/021:55:500
104699129,2cyclictest0-21swapper/021:30:200
104699129,2cyclictest0-21swapper/019:50:180
104699122,7cyclictest0-21swapper/019:41:590
104699121,6cyclictest0-21swapper/020:30:000
104699120,2cyclictest101ktimersoftd/000:32:520
104699120,2cyclictest0-21swapper/023:56:070
104699120,2cyclictest0-21swapper/023:36:470
104699120,2cyclictest0-21swapper/022:00:190
104699120,11cyclictest0-21swapper/023:46:570
104699120,11cyclictest0-21swapper/023:40:210
104699120,11cyclictest0-21swapper/023:19:570
104699120,11cyclictest0-21swapper/022:45:070
104699120,11cyclictest0-21swapper/022:05:390
104699120,11cyclictest0-21swapper/021:40:200
104999118,2cyclictest29748-21kworker/u8:220:43:053
104999118,2cyclictest27423-21kworker/u8:020:31:043
104999118,2cyclictest23131-21kworker/u8:120:14:053
104999118,2cyclictest15212-21kworker/u8:220:07:053
104999118,2cyclictest13015-21kworker/u8:119:51:043
104999111,6cyclictest0-21swapper/319:35:253
104799116,3cyclictest7458-21runrttasks20:50:251
104799112,6cyclictest0-21swapper/120:10:591
104799110,10cyclictest0-21swapper/120:25:161
104699116,3cyclictest7774-21tr20:45:130
104699115,4cyclictest0-21swapper/019:10:590
104699110,10cyclictest0-21swapper/019:15:270
104999100,7cyclictest0-21swapper/320:17:583
104799101,7cyclictest1252-21apt-get20:30:141
104799101,2cyclictest23-21ksoftirqd/120:45:131
104799100,8cyclictest7523-21munin-node1
104699102,7cyclictest0-21swapper/020:19:020
104699102,7cyclictest0-21swapper/019:39:020
104699102,6cyclictest0-21swapper/019:55:590
104699102,6cyclictest0-21swapper/019:30:360
104699101,5cyclictest0-21swapper/020:25:000
104699100,7cyclictest0-21swapper/020:52:590
104699100,3cyclictest24648-21pzem_power20:05:220
10499992,6cyclictest0-21swapper/320:20:593
10479994,3cyclictest19721-21aten2.4_r3power19:55:131
10469994,3cyclictest5098-21sendmail20:40:000
10469992,5cyclictest0-21swapper/020:55:550
10469991,6cyclictest0-21swapper/020:40:250
10469990,6cyclictest0-21swapper/020:30:490
10469973,2cyclictest15666-21aten2.4_r3power19:45:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional