You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 21:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Wed Jul 08, 2026 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1632499974,86cyclictest0-21swapper/009:31:500
1632999967,38cyclictest0-21swapper/109:37:161
16329999621,31cyclictest29405-21gltestperf10:37:061
16329999621,31cyclictest29405-21gltestperf10:37:061
1632499944,26cyclictest0-21swapper/011:31:540
1632499934,30cyclictest0-21swapper/009:53:330
16324999332,54cyclictest21662-21grep11:12:190
1632499923,81cyclictest0-21swapper/010:07:190
1632999915,37cyclictest0-21swapper/111:07:011
1632999915,36cyclictest0-21swapper/111:42:191
1632499915,79cyclictest0-21swapper/009:57:150
1632499915,79cyclictest0-21swapper/009:57:150
1632999894,11cyclictest7573-21munin-node10:52:091
16329998921,9cyclictest19902-21cat07:46:531
1632499894,79cyclictest0-21swapper/009:01:430
1632499893,8cyclictest4292-21netstat10:47:110
1632499893,33cyclictest0-21swapper/007:27:090
1632499875,12cyclictest0-21swapper/009:44:340
1632499874,36cyclictest0-21swapper/011:51:300
1632499874,32cyclictest0-21swapper/009:17:170
1632499873,76cyclictest0-21swapper/011:52:020
1632499873,76cyclictest0-21swapper/010:37:080
1632499873,76cyclictest0-21swapper/010:37:080
1632499873,21cyclictest9070-21diskmemload09:26:140
1632999866,36cyclictest0-21swapper/111:02:141
1632999865,35cyclictest0-21swapper/110:47:011
1632499864,75cyclictest0-21swapper/009:07:170
1632499864,75cyclictest0-21swapper/009:07:170
1632499863,34cyclictest0-21swapper/011:04:530
1632499863,31cyclictest18015-21python11:07:160
1632999854,74cyclictest0-21swapper/110:12:131
16329998520,9cyclictest31789-21latency_hist08:11:531
1632499854,34cyclictest0-21swapper/008:32:050
1632499854,31cyclictest0-21swapper/009:47:010
16324998530,48cyclictest0-21swapper/010:27:150
1632499853,30cyclictest10615-21grep10:57:030
1632499844,32cyclictest0-21swapper/010:12:000
1632499843,38cyclictest0-21swapper/011:58:080
1632499843,38cyclictest0-21swapper/011:58:080
1632499843,34cyclictest0-21swapper/011:44:420
1632999834,34cyclictest0-21swapper/111:17:031
1632999834,34cyclictest0-21swapper/111:17:031
1632999834,27cyclictest0-21swapper/111:38:231
1632499835,12cyclictest0-21swapper/010:17:170
1632499834,32cyclictest0-21swapper/010:02:210
1632499834,31cyclictest0-21swapper/010:42:080
1632499834,30cyclictest0-21swapper/009:12:160
1632499834,30cyclictest0-21swapper/009:12:160
1632499834,28cyclictest16432-21tr08:42:040
1632499834,12cyclictest0-21swapper/011:26:590
16324998330,47cyclictest1716-21grep07:07:150
16324998330,47cyclictest1716-21grep07:07:150
16324998330,10cyclictest30295-21tune2fs08:07:110
1632499833,33cyclictest0-21swapper/010:26:020
1632499833,30cyclictest0-21swapper/006:57:120
1632499833,29cyclictest0-21swapper/010:52:000
1632499833,26cyclictest0-21swapper/008:47:060
1632499833,10cyclictest0-21swapper/009:40:200
16324998325,8cyclictest1227-21runrttasks08:41:190
16324998325,8cyclictest1227-21runrttasks08:41:190
16324998324,8cyclictest17545-21entropy09:32:060
16324998321,11cyclictest0-21swapper/008:52:030
1632999825,34cyclictest0-21swapper/110:31:471
1632999825,31cyclictest0-21swapper/110:57:151
1632999825,14cyclictest1227-21runrttasks10:02:011
1632999824,38cyclictest0-21swapper/108:47:101
16329998219,9cyclictest25952-21cat06:51:551
1632499824,32cyclictest0-21swapper/011:21:570
1632499824,29cyclictest0-21swapper/008:21:520
1632499824,25cyclictest0-21swapper/008:02:140
1632499823,31cyclictest24633-21turbostat07:56:520
1632499822,10cyclictest4890-21ssh11:36:580
16324998219,14cyclictest0-21swapper/009:04:470
1632499815,32cyclictest0-21swapper/006:47:090
1632499814,35cyclictest0-21swapper/007:16:540
1632499814,35cyclictest0-21swapper/007:16:540
1632499814,32cyclictest0-21swapper/010:32:250
1632499813,11cyclictest0-21swapper/011:17:580
1632499813,11cyclictest0-21swapper/011:17:580
159922810,10sleep30-21swapper/307:37:113
1632499804,29cyclictest8444-21tr07:22:060
1632499803,34cyclictest0-21swapper/007:57:150
16324997925,9cyclictest4670-21head08:22:050
16324997920,11cyclictest0-21swapper/006:43:590
219499780,27rtkit-daemon2193-21rtkit-daemon08:20:593
16324997831,41cyclictest26423-21cpuspeed_turbos06:52:090
1632499782,25cyclictest0-21swapper/007:46:530
1632999775,27cyclictest0-21swapper/109:04:591
1632999774,26cyclictest0-21swapper/109:01:201
16329997715,23cyclictest0-21swapper/106:32:091
1632999764,35cyclictest0-21swapper/110:11:481
16324997524,11cyclictest0-21swapper/008:27:210
16324997523,10cyclictest0-21swapper/007:32:300
1632499743,8cyclictest21085-21latency_hist06:41:550
16324997423,8cyclictest32742-21fschecks_time08:12:100
16324997421,11cyclictest0-21swapper/006:32:280
255752720,3sleep2281ktimersoftd/211:19:002
255752720,3sleep2281ktimersoftd/211:19:002
219499710,7rtkit-daemon0-21swapper/307:00:403
1632499703,32cyclictest0-21swapper/007:17:080
219499690,7rtkit-daemon0-21swapper/309:14:573
219499690,7rtkit-daemon0-21swapper/309:14:573
219499680,8rtkit-daemon0-21swapper/306:45:413
219499660,7rtkit-daemon0-21swapper/308:10:153
181112664,13sleep30-21swapper/308:42:463
1633299664,41cyclictest0-21swapper/307:11:583
1633299664,41cyclictest0-21swapper/307:11:583
219499630,8rtkit-daemon0-21swapper/309:28:123
219499630,7rtkit-daemon0-21swapper/309:08:573
219499630,7rtkit-daemon0-21swapper/309:08:573
219499630,7rtkit-daemon0-21swapper/308:03:373
219499630,7rtkit-daemon0-21swapper/307:53:383
219499630,4rtkit-daemon2193-21rtkit-daemon08:59:133
1632499633,14cyclictest0-21swapper/007:47:160
219499620,9rtkit-daemon0-21swapper/308:12:373
219499620,7rtkit-daemon0-21swapper/309:34:563
219499620,7rtkit-daemon0-21swapper/309:23:193
219499620,7rtkit-daemon0-21swapper/307:42:463
219499620,7rtkit-daemon0-21swapper/307:23:093
219499620,6rtkit-daemon0-21swapper/309:18:203
219499620,6rtkit-daemon0-21swapper/307:48:383
219499610,9rtkit-daemon0-21swapper/306:38:113
219499610,7rtkit-daemon0-21swapper/308:22:373
219499610,7rtkit-daemon0-21swapper/307:58:303
219499610,7rtkit-daemon0-21swapper/307:18:473
219499610,7rtkit-daemon0-21swapper/307:09:473
219499610,7rtkit-daemon0-21swapper/307:09:473
219499610,10rtkit-daemon0-21swapper/307:27:313
16331996121,25cyclictest829-21open_files10:42:142
135362614,13sleep30-21swapper/310:12:233
219499600,9rtkit-daemon0-21swapper/306:49:033
219499600,8rtkit-daemon0-21swapper/308:54:133
219499600,7rtkit-daemon0-21swapper/308:47:583
219499600,7rtkit-daemon0-21swapper/308:36:443
219499600,7rtkit-daemon0-21swapper/307:33:313
219499600,3rtkit-daemon0-21swapper/309:05:133
219499600,3rtkit-daemon0-21swapper/306:52:033
214642603,14sleep30-21swapper/310:26:303
166812603,13sleep20-21swapper/206:32:092
1633299604,30cyclictest0-21swapper/309:44:343
146192603,12sleep20-21swapper/210:16:172
219499590,7rtkit-daemon0-21swapper/308:36:593
219499590,7rtkit-daemon0-21swapper/308:36:593
219499590,7rtkit-daemon0-21swapper/308:28:073
219499590,6rtkit-daemon0-21swapper/306:35:113
1633299594,29cyclictest0-21swapper/310:34:593
1633199597,35cyclictest0-21swapper/209:09:582
1633199597,35cyclictest0-21swapper/209:09:582
157792590,5chrt15759-21ssh10:17:063
1633299584,37cyclictest0-21swapper/310:11:483
1633199584,37cyclictest0-21swapper/211:22:042
1633299575,25cyclictest0-21swapper/310:00:183
1633299575,25cyclictest0-21swapper/310:00:183
16332995715,26cyclictest0-21swapper/311:44:583
1632999575,43cyclictest0-21swapper/110:45:121
68582563,45sleep30-21swapper/310:02:403
219499560,5rtkit-daemon2193-21rtkit-daemon07:02:102
174622563,47sleep20-21swapper/211:56:022
1633299565,34cyclictest0-21swapper/309:52:583
16331995617,22cyclictest14680-21ls11:02:172
1632999564,45cyclictest0-21swapper/111:33:581
1632999564,45cyclictest0-21swapper/109:58:381
1632999564,45cyclictest0-21swapper/109:58:381
1632999564,43cyclictest18490-21ssh10:21:571
145772564,13sleep20-21swapper/208:37:492
145772564,13sleep20-21swapper/208:37:492
16332995523,11cyclictest19771-21ssh11:11:463
16332995523,11cyclictest18363-21df11:56:563
16332995523,11cyclictest18363-21df11:56:563
16332995514,25cyclictest21231-21open_inodes11:12:133
1633199557,42cyclictest0-21swapper/209:58:362
1633199557,42cyclictest0-21swapper/209:58:362
16331995539,10cyclictest25734-21grep07:57:122
1633199553,36cyclictest10913-21fschecks_time09:22:072
1632999554,43cyclictest0-21swapper/112:00:121
1632999554,43cyclictest0-21swapper/112:00:121
1633299543,35cyclictest34-21rcuc/311:33:513
1633199548,26cyclictest0-21swapper/208:02:032
1633199543,32cyclictest0-21swapper/207:32:162
1633199543,27cyclictest0-21swapper/209:34:592
16331995423,13cyclictest0-21swapper/209:47:022
16331995416,21cyclictest28303-21latency_hist10:36:462
195432533,43sleep20-21swapper/211:11:362
169512533,7sleep316954-21ssh11:54:103
1633299534,23cyclictest0-21swapper/311:49:173
16332995315,24cyclictest1227-21runrttasks10:38:313
16332995315,24cyclictest1227-21runrttasks10:38:313
1633199535,13cyclictest0-21swapper/209:42:212
1633199534,30cyclictest0-21swapper/208:52:042
1633199534,24cyclictest0-21swapper/209:31:102
1633199533,35cyclictest0-21swapper/210:02:122
1632999533,43cyclictest11513-21ntp_states09:22:171
16329995316,30cyclictest11749-21df_abs11:46:591
38122523,42sleep20-21swapper/210:47:052
285162523,42sleep20-21swapper/208:57:322
1633299527,22cyclictest34-21rcuc/311:37:033
1633299525,28cyclictest0-21swapper/309:47:143
1633299525,15cyclictest0-21swapper/310:44:303
16332995221,13cyclictest0-21swapper/310:57:203
1633199524,34cyclictest0-21swapper/209:56:112
16331995237,9cyclictest1689-21sed07:07:152
16331995237,9cyclictest1689-21sed07:07:152
16331995235,10cyclictest30498-21grep08:07:132
16331995235,10cyclictest15628-21cpuspeed_turbos07:37:062
1633199523,42cyclictest8912-21kernelversion11:42:072
1632999524,43cyclictest0-21swapper/109:14:141
1632999524,43cyclictest0-21swapper/109:14:141
16329995225,9cyclictest17838-21timerwakeupswit08:42:241
88952512,41sleep20-21swapper/208:29:472
209252513,40sleep20-21swapper/211:12:082
192362514,12sleep20-21swapper/208:46:322
1633299515,28cyclictest0-21swapper/307:02:143
1633299515,27cyclictest0-21swapper/309:37:213
1633299515,14cyclictest0-21swapper/311:22:363
1633299515,13cyclictest0-21swapper/311:21:413
1633299515,13cyclictest0-21swapper/311:21:413
1633299514,31cyclictest0-21swapper/310:29:423
16332995121,12cyclictest0-21swapper/311:03:413
1633199514,30cyclictest0-21swapper/211:32:152
1633199514,30cyclictest0-21swapper/208:49:332
1633199514,29cyclictest0-21swapper/209:12:102
1633199514,29cyclictest0-21swapper/209:12:102
1633199514,26cyclictest0-21swapper/208:36:492
1633199513,29cyclictest0-21swapper/210:11:422
16331995120,13cyclictest0-21swapper/207:42:252
16331995116,18cyclictest22990-21ssh10:27:122
1632999516,17cyclictest0-21swapper/111:11:561
1632999514,39cyclictest0-21swapper/109:27:081
1632999513,41cyclictest0-21swapper/111:24:261
16329995121,9cyclictest15876-21cat08:41:511
16329995121,9cyclictest15876-21cat08:41:511
7872506,37sleep10-21swapper/107:07:031
7872506,37sleep10-21swapper/107:07:031
77772502,11sleep20-21swapper/207:20:262
1633299505,28cyclictest0-21swapper/310:52:053
1633299505,26cyclictest31167-21memory11:27:083
1633199508,25cyclictest0-21swapper/209:05:362
1633199505,27cyclictest0-21swapper/208:17:242
1633199504,32cyclictest0-21swapper/210:38:042
1633199504,32cyclictest0-21swapper/210:38:042
1633199504,29cyclictest0-21swapper/211:27:162
1633199504,28cyclictest0-21swapper/210:52:062
1633199504,28cyclictest0-21swapper/210:17:052
16331995035,9cyclictest12305-21grep07:27:272
16331995032,11cyclictest5222-21tune2fs11:37:022
1633199503,37cyclictest22174-21users09:37:212
16331995020,12cyclictest0-21swapper/210:22:082
1633199502,35cyclictest0-21swapper/211:01:392
1632999505,27cyclictest0-21swapper/110:17:071
1632999504,38cyclictest10785-21python08:32:181
16329995018,26cyclictest7870-21grep09:17:131
1633299494,27cyclictest0-21swapper/310:46:573
16331994935,9cyclictest18764-21gltestperf11:57:012
16331994935,9cyclictest18764-21gltestperf11:57:012
16331994920,12cyclictest0-21swapper/211:47:542
1632999494,39cyclictest0-21swapper/111:27:131
1632999494,38cyclictest0-21swapper/109:55:461
1632999494,13cyclictest0-21swapper/108:52:231
1632999494,12cyclictest0-21swapper/110:32:171
1632999493,39cyclictest23745-21tail09:41:591
101732492,10sleep20-21swapper/207:25:182
259432483,39sleep126017-21munin-node07:57:171
219499480,4rtkit-daemon2193-21rtkit-daemon06:27:562
219499480,4rtkit-daemon2193-21rtkit-daemon06:27:562
1633199484,28cyclictest0-21swapper/208:16:502
1633199484,27cyclictest0-21swapper/207:52:212
1633199484,26cyclictest0-21swapper/209:17:092
1633199484,26cyclictest0-21swapper/208:22:222
16331994834,8cyclictest20689-21grep06:37:302
1633199483,28cyclictest0-21swapper/206:43:332
16331994817,13cyclictest0-21swapper/207:12:082
16331994817,13cyclictest0-21swapper/207:12:082
1632999485,36cyclictest29795-21rm09:50:281
16329994810,31cyclictest0-21swapper/111:52:021
102022485,37sleep10-21swapper/107:25:411
219572472,9sleep20-21swapper/207:48:192
1633199474,26cyclictest0-21swapper/206:56:552
1633199474,25cyclictest0-21swapper/207:00:052
1632999474,27cyclictest0-21swapper/107:47:031
1632999474,26cyclictest0-21swapper/109:11:481
1632999474,26cyclictest0-21swapper/109:11:481
16329994720,9cyclictest7941-21cat07:21:541
233562463,36sleep10-21swapper/106:45:301
1632999462,38cyclictest0-21swapper/109:33:511
1632499464,36cyclictest0-21swapper/007:37:040
141412464,36sleep00-21swapper/006:27:010
141412464,36sleep00-21swapper/006:27:010
1632999459,31cyclictest0-21swapper/108:31:511
1632999459,30cyclictest0-21swapper/106:52:081
1632999455,34cyclictest0-21swapper/106:37:171
1632999454,35cyclictest0-21swapper/107:00:541
1632999454,33cyclictest0-21swapper/107:12:051
1632999454,33cyclictest0-21swapper/107:12:051
1632999453,11cyclictest0-21swapper/108:22:241
16331994420,17cyclictest24632-21wc06:47:172
1632999444,34cyclictest0-21swapper/107:41:531
1632999444,32cyclictest0-21swapper/108:06:531
16329994430,8cyclictest12239-21tail07:27:261
162752444,34sleep10-21swapper/106:31:401
162752444,34sleep10-21swapper/106:31:401
1632999434,34cyclictest0-21swapper/107:52:091
320122423,33sleep10-21swapper/108:11:561
1632999424,32cyclictest0-21swapper/107:32:241
1632999424,31cyclictest0-21swapper/107:02:081
1632999423,32cyclictest0-21swapper/108:21:521
1632499423,33cyclictest0-21swapper/007:04:070
1633299382,23cyclictest0-21swapper/306:31:573
1633299382,23cyclictest0-21swapper/306:31:573
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional