You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-13 - 23:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot7s.osadl.org (updated Mon Jul 13, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3199029644,47sleep00-21swapper/007:05:250
32028958,25sleep30-21swapper/307:08:323
3183428460,19sleep20-21swapper/207:05:222
3269827958,6sleep10-21swapper/107:07:221
27392660,0sleep10-21swapper/110:32:101
292822640,1sleep10-21swapper/110:48:001
56199284,1cyclictest25521-21diskmemload10:38:263
56199280,2cyclictest9835-21turbostat09:20:003
56199280,26cyclictest0-21swapper/309:40:283
266122270,0sleep00-21swapper/011:44:580
56199262,1cyclictest25521-21diskmemload12:35:063
64842250,0sleep00-21swapper/011:13:080
56199256,18cyclictest25521-21diskmemload12:05:253
56199254,1cyclictest25521-21diskmemload10:41:273
56199251,2cyclictest25521-21diskmemload10:33:183
56199250,2cyclictest21343-21/usr/sbin/munin10:05:243
56199244,3cyclictest25521-21diskmemload09:27:313
56199241,18cyclictest0-21swapper/312:20:183
56199240,3cyclictest0-21swapper/311:37:543
559992420,3cyclictest55750irq/128-enp2s0-10:03:112
548992422,1cyclictest1162-21cpu11:10:111
56199232,1cyclictest25521-21diskmemload09:50:373
561992312,9cyclictest0-21swapper/312:30:253
56199230,5cyclictest0-21swapper/311:50:163
559992320,3cyclictest55550irq/126-enp2s0-09:52:242
559992320,2cyclictest55850irq/129-enp2s0-11:01:222
559992320,2cyclictest55850irq/129-enp2s0-10:53:422
559992320,2cyclictest55750irq/128-enp2s0-09:27:002
559992319,3cyclictest55850irq/129-enp2s0-09:23:262
559992319,3cyclictest55650irq/127-enp2s0-11:16:562
55999231,1cyclictest25077-21apt-get11:05:132
548992320,1cyclictest0-21swapper/108:55:271
548992319,3cyclictest55750irq/128-enp2s0-12:07:421
54899231,20cyclictest0-21swapper/107:55:141
56199223,2cyclictest28965-21switchtime11:45:273
561992220,2cyclictest55750irq/128-enp2s0-10:17:373
561992220,1cyclictest55550irq/126-enp2s0-11:12:323
561992219,2cyclictest55650irq/127-enp2s0-09:33:203
56199221,4cyclictest25521-21diskmemload10:55:283
559992221,1cyclictest0-21swapper/211:14:562
559992219,2cyclictest55850irq/129-enp2s0-10:45:182
559992219,2cyclictest55750irq/128-enp2s0-11:36:052
559992219,2cyclictest55750irq/128-enp2s0-10:57:522
559992219,2cyclictest55650irq/127-enp2s0-10:44:312
559992219,2cyclictest55550irq/126-enp2s0-09:15:482
559992218,3cyclictest55850irq/129-enp2s0-10:37:502
559992218,3cyclictest55850irq/129-enp2s0-09:44:232
559992218,2cyclictest55750irq/128-enp2s0-09:58:092
55999220,22cyclictest0-21swapper/212:07:542
55999220,22cyclictest0-21swapper/209:49:442
548992218,3cyclictest55750irq/128-enp2s0-12:19:481
561992119,1cyclictest55550irq/126-enp2s0-10:52:113
56199210,2cyclictest0-21swapper/309:13:053
559992119,1cyclictest55750irq/128-enp2s0-11:46:392
559992118,3cyclictest55650irq/127-enp2s0-10:28:282
559992118,3cyclictest55650irq/127-enp2s0-09:12:492
559992118,2cyclictest55850irq/129-enp2s0-12:14:312
559992118,2cyclictest55850irq/129-enp2s0-10:10:552
559992118,2cyclictest55750irq/128-enp2s0-11:22:302
559992117,3cyclictest55850irq/129-enp2s0-10:05:162
559992117,3cyclictest55750irq/128-enp2s0-11:26:332
559992117,3cyclictest55750irq/128-enp2s0-09:31:132
559992117,3cyclictest55650irq/127-enp2s0-12:39:582
55999210,20cyclictest0-21swapper/210:16:302
561992017,2cyclictest55550irq/126-enp2s0-12:00:393
561992017,2cyclictest55550irq/126-enp2s0-11:42:193
56199200,3cyclictest25112-21unixbench_singl09:05:263
56199200,3cyclictest0-21swapper/309:46:323
559992017,2cyclictest55650irq/127-enp2s0-11:53:392
559992017,2cyclictest55550irq/126-enp2s0-09:36:012
559992016,3cyclictest55850irq/129-enp2s0-12:16:462
559992016,3cyclictest55750irq/128-enp2s0-11:42:252
55999200,20cyclictest0-21swapper/212:32:312
548992018,1cyclictest25521-21diskmemload11:00:591
54899200,1cyclictest0-21swapper/108:00:151
54699200,1cyclictest0-21swapper/011:52:070
56199194,1cyclictest25521-21diskmemload10:02:363
56199192,4cyclictest7699-21turbostat.cron07:25:003
561991916,2cyclictest55650irq/127-enp2s0-09:35:023
56199190,3cyclictest0-21swapper/311:30:133
559991917,1cyclictest55850irq/129-enp2s0-12:00:272
55999190,16cyclictest0-21swapper/210:22:232
548991918,1cyclictest55750irq/128-enp2s0-12:30:511
548991918,1cyclictest0-21swapper/112:02:291
548991917,1cyclictest55850irq/129-enp2s0-12:20:151
548991916,2cyclictest55750irq/128-enp2s0-12:11:141
54899190,18cyclictest27899-21sensors11:25:251
54699195,8cyclictest0-21swapper/010:50:120
54699190,1cyclictest0-21swapper/010:55:040
56199183,6cyclictest21022-21apt-get09:00:143
561991815,2cyclictest55550irq/126-enp2s0-12:25:393
561991815,2cyclictest55550irq/126-enp2s0-11:27:483
561991815,2cyclictest39-21ksoftirqd/310:10:023
56199181,4cyclictest3646-21unixbench_multi08:20:273
56199181,2cyclictest0-21swapper/311:55:513
55999186,7cyclictest55650irq/127-enp2s0-11:55:142
559991812,3cyclictest21014-21apache_processe12:20:112
548991814,2cyclictest0-21swapper/111:05:501
54899180,1cyclictest0-21swapper/110:11:001
54699182,15cyclictest0-21swapper/008:15:160
546991817,1cyclictest0-21swapper/008:06:030
56199173,8cyclictest13525-21apt-get10:20:023
56199172,2cyclictest30673-21sed08:10:193
56199172,2cyclictest0-21swapper/311:17:233
56199171,5cyclictest26885-21timerwakeupswit11:05:273
56199171,4cyclictest24824-21apt-get08:00:153
561991714,2cyclictest55550irq/126-enp2s0-10:46:133
56199171,3cyclictest8217-21timerandwakeup08:30:283
56199171,3cyclictest8217-21timerandwakeup08:30:273
56199170,4cyclictest0-21swapper/310:28:043
56199170,0cyclictest0-21swapper/311:02:283
559991713,3cyclictest55550irq/126-enp2s0-10:30:242
548991715,1cyclictest55850irq/129-enp2s0-11:32:171
548991710,2cyclictest225-21systemd-journal12:28:331
54899170,0cyclictest0-21swapper/111:21:241
546991712,3cyclictest13-21ksoftirqd/011:20:300
54699170,0cyclictest0-21swapper/011:38:110
56199169,6cyclictest6229-21grep09:55:273
56199167,4cyclictest4310-21ssh12:10:113
56199164,4cyclictest225-21systemd-journal07:44:593
561991613,2cyclictest55650irq/127-enp2s0-09:22:353
56199160,2cyclictest0-21swapper/312:18:003
559991614,2cyclictest55850irq/129-enp2s0-12:28:592
548991614,1cyclictest55750irq/128-enp2s0-11:59:001
54899160,2cyclictest0-21swapper/109:49:281
546991616,0cyclictest0-21swapper/009:29:450
54699160,12cyclictest0-21swapper/011:17:330
56199159,5cyclictest5583-21tr08:25:223
56199150,1cyclictest0-21swapper/307:25:173
56199150,14cyclictest23787-21readlink07:55:243
559991511,3cyclictest18360-21turbostat08:55:012
54899154,9cyclictest28334-21irqstats11:45:211
548991513,1cyclictest27844-21ssh09:30:161
54899150,3cyclictest0-21swapper/109:44:211
54899150,15cyclictest0-21swapper/110:57:421
54699151,8cyclictest20539-21sendmail12:20:000
54699150,4cyclictest0-21swapper/009:15:160
54699150,1cyclictest0-21swapper/009:45:420
54699150,14cyclictest21761-21wc12:00:240
56199148,5cyclictest17759-21cut11:20:143
56199148,5cyclictest1624-21taskset08:19:093
56199148,5cyclictest13458-21grep07:35:173
56199147,6cyclictest11136-21taskset08:39:083
56199145,7cyclictest0-21swapper/307:10:173
561991410,3cyclictest15987-21turbostat08:50:003
559991411,2cyclictest55650irq/127-enp2s0-11:31:002
559991410,3cyclictest6360-21grep07:20:162
548991412,2cyclictest38150irq/131-snd_hda09:14:221
548991412,1cyclictest20502-21ssh11:40:251
54899140,1cyclictest0-21swapper/109:55:281
54699148,4cyclictest26234-21ssh11:05:200
54699146,7cyclictest11505-21apt-get08:40:130
54699146,4cyclictest27579-21apache212:05:170
546991413,1cyclictest0-21swapper/010:28:450
546991410,3cyclictest31448-21wc10:10:250
546991410,3cyclictest1501-21basename09:14:590
54699140,8cyclictest24563-21sendmail07:59:590
54699140,1cyclictest0-21swapper/009:24:560
56199138,4cyclictest13420-21taskset08:42:423
56199137,5cyclictest10900-21chrt07:30:153
561991310,2cyclictest20701-21sort07:50:183
559991310,2cyclictest30383-21sed08:10:162
559991310,2cyclictest21793-21grep09:00:182
559991310,2cyclictest16090-21head07:40:162
54899139,3cyclictest2030-21head07:10:221
54899134,5cyclictest225-21systemd-journal07:20:001
54899131,1cyclictest15687-21users12:35:271
548991310,2cyclictest22492-21ssh10:05:221
54899130,12cyclictest20101-21ssh10:04:351
54899130,10cyclictest7785-21ssh10:15:321
54699139,3cyclictest4078-21ssh11:30:290
54699139,3cyclictest20276-21cut07:50:160
54699139,3cyclictest19514-21cron12:00:000
54699134,4cyclictest225-21systemd-journal10:45:010
54699131,6cyclictest26952-21perf08:05:010
546991311,1cyclictest28661-21ssh12:24:480
546991311,1cyclictest27571-21apache211:29:010
546991310,2cyclictest5216-21grep12:10:170
546991310,2cyclictest31857-21cat10:50:000
546991310,2cyclictest25615-21wc09:10:010
546991310,2cyclictest22242-21grep10:05:190
546991310,2cyclictest21667-21sort09:00:170
54699130,8cyclictest24377-21basename11:04:590
54699130,2cyclictest0-21swapper/010:01:580
54699130,1cyclictest0-21swapper/009:57:120
56199129,2cyclictest3433-21tr07:15:163
56199129,2cyclictest27335-21grep08:05:153
56199129,2cyclictest19300-21cut08:55:173
56199124,2cyclictest17900-21tr07:45:163
56199120,4cyclictest225-21systemd-journal08:55:013
55999129,2cyclictest4100-21munin-plugin-st08:25:012
55999129,2cyclictest26428-21grep08:00:282
55999129,2cyclictest22664-21grep07:55:152
55999129,2cyclictest15104-21head08:45:232
55999128,3cyclictest21471-21ls07:50:262
54899129,2cyclictest23235-21wc09:05:001
54899129,2cyclictest16115-21grep10:40:151
54899129,2cyclictest12926-21grep07:35:151
54899128,3cyclictest5373-21ssh09:16:311
54899128,3cyclictest306-21sed08:15:181
54899120,11cyclictest0-21swapper/111:50:021
54699129,2cyclictest7410-21cut08:30:180
54699129,2cyclictest7410-21cut08:30:170
54699129,2cyclictest30362-21grep08:10:150
54699129,2cyclictest2631-21cron11:50:000
54699129,2cyclictest17480-21cat07:45:000
54699129,2cyclictest14531-21head12:35:170
54699129,2cyclictest11068-21dump-pmu-power09:39:590
54699128,3cyclictest13734-21mktemp10:20:010
54699128,2cyclictest2094-21grep08:20:140
54699121,7cyclictest0-21swapper/009:34:210
54699120,1cyclictest0-21swapper/012:29:060
54699120,1cyclictest0-21swapper/009:52:460
55999118,2cyclictest9113-21ntp_states07:25:232
55999118,2cyclictest856-21wget07:10:142
55999118,2cyclictest3939-21head07:15:192
55999118,2cyclictest27193-21apache_accesses08:05:122
55999118,2cyclictest24122-21grep09:05:172
55999118,2cyclictest19333-21sed07:45:282
55999113,6cyclictest12175-21grep08:40:162
55999110,10cyclictest0-21swapper/208:55:142
54899119,1cyclictest23726-21ssh09:27:391
54899118,2cyclictest7099-21cat08:30:151
54899118,2cyclictest7099-21cat08:30:141
54899118,2cyclictest6616-21cat08:30:011
54899118,2cyclictest28637-21ssh10:28:351
54899118,2cyclictest28147-21ssh09:50:041
54899118,2cyclictest27215-21cut08:05:141
54899118,2cyclictest25480-21seq09:09:551
54899118,2cyclictest2126-21ssh10:50:361
54899118,2cyclictest16811-21ssh10:20:531
54899118,2cyclictest12257-21tail08:40:161
54899118,2cyclictest11740-21cat09:20:191
54899118,2cyclictest11706-21wc11:35:181
54899116,4cyclictest3745-21taskset08:20:371
54899116,4cyclictest3127-21ssh09:35:081
54899116,3cyclictest17405-21cron07:44:591
54899110,10cyclictest9171-21ssh11:15:061
54699118,2cyclictest9665-21ssh10:35:530
54699118,2cyclictest9191-21head07:25:240
54699118,2cyclictest5860-21tr07:20:140
54699118,2cyclictest5446-21ssh10:34:180
54699118,2cyclictest5312-21perf07:20:000
54699118,2cyclictest18369-21sh08:55:000
54699118,2cyclictest18224-21df07:45:160
54699118,2cyclictest16185-21head10:20:260
54699118,2cyclictest14818-21systemctl07:40:000
54699118,2cyclictest14640-21grep08:45:180
54699118,2cyclictest12606-21cat09:40:200
54699118,2cyclictest11548-21grep07:30:220
54699110,10cyclictest6024-21rm12:30:180
55999108,1cyclictest9081-21basename08:35:112
55999108,1cyclictest85942sleep208:31:582
55999108,1cyclictest85942sleep208:31:572
55999108,1cyclictest32670-21tune2fs08:15:182
55999108,1cyclictest14458-21taskset07:35:412
55999107,2cyclictest4959-21fschecks_count08:25:182
55999100,9cyclictest11294-21cat07:30:202
54899108,1cyclictest219392sleep107:52:221
54899107,2cyclictest9564-21cat08:35:141
54899107,2cyclictest31598-21taskset08:13:251
54899107,2cyclictest10406-21cut07:30:141
54899103,3cyclictest225-21systemd-journal07:45:001
54899102,7cyclictest7084-21ssh10:35:121
54899100,9cyclictest8541-21cat07:25:181
54899100,9cyclictest17273-21missed_timers08:50:211
54899100,9cyclictest14322-21cat08:45:161
54699108,1cyclictest11029-21chrt08:37:440
54699107,2cyclictest4854-21grep08:25:160
54699107,2cyclictest19455-21basename08:55:190
54699107,2cyclictest1658-21if_enp0s31f607:10:170
5489992,3cyclictest225-21systemd-journal07:20:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional