You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-21 - 05:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot7s.osadl.org (updated Fri Aug 21, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1053329381,8sleep00-21swapper/019:06:370
1067728964,8sleep20-21swapper/219:08:292
1067128265,13sleep10-21swapper/119:08:231
1044928269,9sleep30-21swapper/319:05:323
1088599250,5cyclictest0-21swapper/221:40:132
1088599250,24cyclictest0-21swapper/200:14:392
1087999250,1cyclictest0-21swapper/022:40:180
10879992323,0cyclictest0-21swapper/021:55:500
10885992221,1cyclictest0-21swapper/222:47:582
1088299220,22cyclictest0-21swapper/122:57:011
1088299220,22cyclictest0-21swapper/121:55:251
10879992221,1cyclictest0-21swapper/023:38:380
10879992221,1cyclictest0-21swapper/000:35:350
10879992221,1cyclictest0-21swapper/000:04:590
1088599210,21cyclictest0-21swapper/221:36:382
1088599210,20cyclictest0-21swapper/223:55:302
10879992120,1cyclictest0-21swapper/022:48:510
1088599200,20cyclictest0-21swapper/223:03:412
10882992018,1cyclictest0-21swapper/119:30:111
1087999200,19cyclictest0-21swapper/000:33:490
1088999190,18cyclictest0-21swapper/323:03:403
1088299190,1cyclictest0-21swapper/123:20:251
10879991918,1cyclictest0-21swapper/022:26:020
10879991918,1cyclictest0-21swapper/021:35:250
10879991918,1cyclictest0-21swapper/000:19:290
1088999181,16cyclictest26892-21runrttasks22:20:583
1088599180,17cyclictest0-21swapper/223:34:302
10882991814,3cyclictest3405-21diskmemload21:35:581
1088299180,1cyclictest0-21swapper/121:20:341
1088299180,15cyclictest0-21swapper/100:00:481
1088999173,14cyclictest0-21swapper/322:45:063
1088999170,2cyclictest0-21swapper/323:38:473
1088999170,2cyclictest0-21swapper/321:24:283
1088299170,5cyclictest0-21swapper/123:44:341
10879991715,2cyclictest0-21swapper/023:41:450
1087999170,14cyclictest0-21swapper/021:40:130
10889991615,1cyclictest0-21swapper/322:30:253
10889991613,1cyclictest0-21swapper/323:05:453
1088999161,1cyclictest15307-21df_inode21:55:153
1088999160,7cyclictest0-21swapper/323:17:263
1088999160,15cyclictest0-21swapper/323:34:303
1088999160,13cyclictest0-21swapper/321:49:003
1088599165,7cyclictest14686-21apt-get21:55:132
10885991614,1cyclictest0-21swapper/200:18:162
1088599160,1cyclictest0-21swapper/200:33:332
1088299162,11cyclictest3405-21diskmemload21:51:161
1087999160,15cyclictest0-21swapper/000:14:080
1088999159,5cyclictest24755-21cron22:39:593
1088999150,2cyclictest0-21swapper/323:50:223
1088999150,13cyclictest0-21swapper/322:14:243
10885991514,1cyclictest0-21swapper/221:21:252
1088599150,14cyclictest0-21swapper/221:10:032
1088299157,6cyclictest38150irq/131-snd_hda00:35:151
1088299151,1cyclictest0-21swapper/100:28:571
1087999152,7cyclictest26314-21munin-run20:50:000
10879991513,1cyclictest0-21swapper/000:08:450
1087999150,15cyclictest0-21swapper/000:22:330
1088999145,5cyclictest6392-21cut00:04:593
10885991410,3cyclictest0-21swapper/223:52:032
10885991410,2cyclictest10870-21cyclictest22:41:122
1088599140,14cyclictest0-21swapper/223:49:582
1088299142,8cyclictest9521-21sh00:05:511
1087999140,1cyclictest0-21swapper/022:19:450
23572130,1sleep20-21swapper/200:20:482
1088999139,3cyclictest4112-21ssh22:07:503
1088999134,4cyclictest0-21swapper/322:15:143
10889991310,2cyclictest8110-21cat22:30:003
10889991310,2cyclictest5241-21cut21:30:153
10889991310,2cyclictest4824-21sh20:05:013
10889991310,2cyclictest19236-21ntpq22:55:203
10889991310,2cyclictest17612-21head00:30:183
1088999130,1cyclictest0-21swapper/323:12:493
1088999130,12cyclictest30137-21grep21:25:203
10885991311,1cyclictest0-21swapper/221:25:222
10885991310,2cyclictest30392-21wc20:55:252
1088599130,9cyclictest0-21swapper/223:08:242
1088599130,13cyclictest0-21swapper/221:49:002
1088599130,12cyclictest29193-21cat23:20:252
1088299139,2cyclictest0-21swapper/122:11:191
1088299132,6cyclictest2426-21cron20:00:001
1088299131,5cyclictest3848-21cat23:05:231
10882991310,2cyclictest5324-21tr20:05:151
10882991310,2cyclictest26778-21ssh23:00:151
10882991310,2cyclictest1814-21wc22:25:251
10882991310,2cyclictest16408-21turbostat22:35:001
10882991310,2cyclictest14937-21tr20:25:141
10882991310,2cyclictest11417-21wc20:15:251
10882991310,1cyclictest0-21swapper/122:20:161
1088299130,0cyclictest0-21swapper/123:32:331
1087999138,3cyclictest3405-21diskmemload23:17:030
1087999135,7cyclictest30152-21munin-run00:00:010
10879991312,1cyclictest0-21swapper/022:07:340
10879991311,2cyclictest0-21swapper/023:12:100
10879991310,2cyclictest14094-21grep19:15:170
1087999130,9cyclictest0-21swapper/022:04:550
1088999129,2cyclictest31060-21dump-pmu-power21:00:003
1088999129,2cyclictest31050-21uniq19:50:173
1088999129,2cyclictest29102-21cut19:45:193
1088999129,2cyclictest26449-21cat22:40:203
1088999129,2cyclictest25263-21tr00:15:223
1088999129,2cyclictest23926-21cron20:45:003
1088999129,2cyclictest20156-21grep20:35:163
1088999129,2cyclictest16369-21grep20:25:253
1088999129,2cyclictest14222-21cat21:55:013
1088999128,3cyclictest30660-21sh23:21:263
1088999127,4cyclictest1677-21cat00:20:273
1088999122,4cyclictest225-21systemd-journal20:00:013
10889991210,1cyclictest7952-21ls00:05:203
10889991210,1cyclictest29360-21cut23:40:143
1088599129,2cyclictest8676-21tr00:25:152
1088599129,2cyclictest4367-21tail20:00:272
1088599129,2cyclictest30746-21cut00:00:132
1088599129,2cyclictest20150-21apt21:20:002
1088599129,2cyclictest15448-21cut20:25:162
10885991210,1cyclictest7142-21cut21:50:182
10885991210,1cyclictest27114-21grep00:35:262
10885991210,1cyclictest11959-21tr23:10:202
1088599120,12cyclictest0-21swapper/222:59:082
1088599120,11cyclictest7075-21sed21:30:282
1088299129,2cyclictest5823-21grep21:10:251
1088299129,2cyclictest499-21sed21:00:241
1088299129,2cyclictest32490-21grep19:55:011
1088299129,2cyclictest30944-21cron22:05:001
1088299129,2cyclictest22801-21sed21:40:171
1088299129,2cyclictest19855-21tail23:15:171
1088299129,2cyclictest17560-21tail19:20:251
1088299129,2cyclictest16356-21grep00:10:171
1088299129,2cyclictest1547-21grep22:45:161
1088299129,2cyclictest13995-21grep23:50:151
1088299128,3cyclictest8123-21ssh23:27:371
1088299128,3cyclictest8123-21ssh23:27:371
1088299128,3cyclictest13867-21kworker/u8:122:05:481
1088299128,3cyclictest11014-21grep22:50:211
1088299120,11cyclictest29662-21tail20:55:171
1088299120,11cyclictest0-21swapper/121:45:191
1088299120,10cyclictest3405-21diskmemload23:59:591
1087999129,2cyclictest5862-21sh21:50:010
1087999129,2cyclictest24852-21grep19:35:230
1087999129,2cyclictest15460-21grep20:25:150
1087999129,2cyclictest14733-21head23:50:180
1087999124,7cyclictest9882-21apache221:24:340
10879991211,1cyclictest0-21swapper/021:19:030
1087999120,11cyclictest6252-21grep23:45:190
1087999120,11cyclictest4189-21grep21:10:140
1088999119,1cyclictest5719-21grep23:45:153
1088999119,1cyclictest32095-21sed21:00:173
1088999119,1cyclictest22492-21sed19:30:253
1088999119,1cyclictest12340-21grep20:20:123
1088999118,3cyclictest55650irq/127-enp2s0-00:28:583
1088999118,2cyclictest8337-21cat20:10:173
1088999118,2cyclictest6574-21cat20:05:253
1088999118,2cyclictest3465-21ssh23:25:143
1088999118,2cyclictest3465-21ssh23:25:133
1088999118,2cyclictest30976-21basename22:05:013
1088999118,2cyclictest25962-21grep00:35:183
1088999118,2cyclictest25732-21ssh23:56:373
1088999118,2cyclictest18840-21taskset20:31:003
1088999118,2cyclictest14744-21sh21:15:463
1088999118,2cyclictest11289-21grep20:15:223
1088999116,2cyclictest15548-21apt-get00:10:123
1088999115,3cyclictest5894-21head21:10:263
1088599119,2cyclictest31984-21cut21:00:152
1088599119,1cyclictest995-21head22:25:172
1088599119,1cyclictest9383-21tail22:10:282
1088599119,1cyclictest8829-21cut22:30:142
1088599119,1cyclictest21413-21fschecks_time23:35:182
1088599119,1cyclictest1741-21wc19:55:242
1088599119,1cyclictest17241-21taskset22:35:142
1088599119,1cyclictest13089-21sed20:20:162
1088599118,2cyclictest9593-21awk20:15:012
1088599118,2cyclictest7100-21chrt00:05:162
1088599118,2cyclictest4184-21cut23:25:162
1088599118,2cyclictest4184-21cut23:25:162
1088599118,2cyclictest31386-21ssh22:05:112
1088599118,2cyclictest30572-21cat19:50:152
1088599118,2cyclictest29507-21tr23:40:152
1088599118,2cyclictest23781-21sed19:35:132
1088599118,2cyclictest22424-21wc19:30:242
1088599118,2cyclictest18715-21ssh23:15:002
1088599118,2cyclictest17605-21grep19:20:252
1088599118,2cyclictest12544-21sed19:10:242
1088599118,2cyclictest11287-21sort22:50:252
1088599113,6cyclictest15606-21sh19:19:592
1088599111,9cyclictest3405-21diskmemload22:23:142
1088599110,10cyclictest16329-21expr22:15:142
1088299118,2cyclictest7062-21sed21:30:271
1088299118,2cyclictest6683-21ntpq23:45:221
1088299118,2cyclictest29330-21ssh00:18:041
1088299118,2cyclictest28673-21cron20:55:001
1088299118,2cyclictest25606-21grep22:40:141
1088299118,2cyclictest25259-21wc20:45:201
1088299118,2cyclictest23353-21ssh22:38:551
1088299118,2cyclictest22512-21sh00:33:201
1088299118,2cyclictest21740-21sh22:18:301
1088299118,2cyclictest20470-21cat19:30:001
1088299118,2cyclictest1902-21grep00:20:281
1088299118,2cyclictest17839-21cut20:30:181
1088299118,2cyclictest15112-21sed19:15:261
1088299118,2cyclictest14833-21ssh21:15:501
1088299118,2cyclictest12854-21grep20:20:141
1088299117,3cyclictest25-21ksoftirqd/121:05:161
1088299116,2cyclictest30632-21open_files21:25:241
1088299111,9cyclictest0-21swapper/120:00:131
1088299111,7cyclictest0-21swapper/119:35:111
1087999119,1cyclictest9577-21latency_hist20:15:000
1087999119,1cyclictest26459-21tr23:00:150
1087999119,1cyclictest13880-21grep23:30:260
1087999119,1cyclictest10527-21grep22:30:260
1087999119,1cyclictest1011-21cron21:04:590
1087999118,2cyclictest9930-21taskset23:09:480
1087999118,2cyclictest7221-21ssh22:10:050
1087999118,2cyclictest32219-21ssh21:26:100
1087999118,2cyclictest29231-21tr20:55:140
1087999118,2cyclictest29090-21cat19:45:180
1087999118,2cyclictest26850-21tr20:50:140
1087999118,2cyclictest22678-21taskset19:31:280
1087999118,2cyclictest22580-21sed20:40:180
1087999118,2cyclictest19158-21ssh22:35:380
1087999118,2cyclictest18186-21cat19:25:010
1087999116,1cyclictest3405-21diskmemload21:54:560
1087999112,8cyclictest0-21swapper/020:15:120
1087999110,3cyclictest0-21swapper/021:33:210
1088999108,1cyclictest26623-21cat19:40:173
1088999108,1cyclictest25089-21sh21:41:203
1088999108,1cyclictest18386-21hostname19:25:153
1088999108,1cyclictest16564-21grep19:20:153
1088999108,1cyclictest13624-21apache_processe21:35:143
1088999108,1cyclictest110792sleep319:10:093
1088999107,2cyclictest26328-21sleep20:49:593
1088999107,2cyclictest13916-21expr19:15:153
1088999106,2cyclictest26862-21tr20:50:143
1088999100,9cyclictest1365-21wget21:05:143
1088999100,9cyclictest11218-21sensors22:50:243
1088599108,1cyclictest3233-21chrt21:07:562
1088599108,1cyclictest28273-21grep20:50:262
1088599108,1cyclictest25682-21wc20:45:262
1088599108,1cyclictest24980-21date22:00:262
1088599108,1cyclictest22167-21expr20:40:152
1088599108,1cyclictest10048-21wc20:15:132
1088599107,2cyclictest5150-21aten2_r7power_e20:05:142
1088599107,2cyclictest26733-21cut19:40:182
1088599107,2cyclictest18721-21expr19:25:162
1088599103,4cyclictest225-21systemd-journal19:45:112
1088299108,1cyclictest22090-21ssh23:35:231
1088299108,1cyclictest21481-21taskset20:39:261
1088299108,1cyclictest104852sleep123:10:051
1088299108,1cyclictest0-21swapper/119:40:121
1088299107,2cyclictest11181-21grep19:10:131
1088299105,2cyclictest225-21systemd-journal19:45:111
1087999108,2cyclictest9378-21ssh22:50:050
1087999108,2cyclictest21333-21taskset20:37:300
1087999108,1cyclictest5815-21head20:05:180
1087999108,1cyclictest14261-21chrt20:23:110
1087999107,2cyclictest8072-21sh00:25:010
1087999107,2cyclictest5419-21ssh23:25:300
1087999107,2cyclictest5419-21ssh23:25:290
1087999107,2cyclictest2834-21basename20:00:140
1087999107,2cyclictest23945-21ssh22:20:020
1087999107,2cyclictest11884-21head19:10:160
1087999102,5cyclictest2443-21ls20:00:000
1087999100,9cyclictest27720-21expr23:20:140
1087999100,9cyclictest25626-21taskset19:40:120
1087999100,9cyclictest18709-21ls19:25:160
1087999100,9cyclictest17869-21ssh22:55:100
108899998,1cyclictest24391-21ntpq19:35:193
108859997,2cyclictest21435-21chrt20:38:512
108859996,2cyclictest17297-21tr20:30:152
108829997,1cyclictest22621-21sh20:40:181
108829996,2cyclictest8115-21cut20:10:151
108799997,1cyclictest30326-21chrt19:50:010
108799997,1cyclictest17701-21fschecks_time20:30:170
108799997,1cyclictest1489-21cpuspeed_turbos21:05:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional