From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lists.gentoo.org (pigeon.gentoo.org [208.92.234.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by finch.gentoo.org (Postfix) with ESMTPS id 70FCB138AE9 for ; Fri, 29 Dec 2017 17:18:49 +0000 (UTC) Received: from pigeon.gentoo.org (localhost [127.0.0.1]) by pigeon.gentoo.org (Postfix) with SMTP id 4E518E0F33; Fri, 29 Dec 2017 17:18:46 +0000 (UTC) Received: from smtp.gentoo.org (dev.gentoo.org [IPv6:2001:470:ea4a:1:5054:ff:fec7:86e4]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by pigeon.gentoo.org (Postfix) with ESMTPS id F034FE0F33 for ; Fri, 29 Dec 2017 17:18:45 +0000 (UTC) Received: from oystercatcher.gentoo.org (oystercatcher.gentoo.org [148.251.78.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.gentoo.org (Postfix) with ESMTPS id 640C233BF05 for ; Fri, 29 Dec 2017 17:18:43 +0000 (UTC) Received: from localhost.localdomain (localhost [IPv6:::1]) by oystercatcher.gentoo.org (Postfix) with ESMTP id EDA65AE8F for ; Fri, 29 Dec 2017 17:18:41 +0000 (UTC) From: "Alice Ferrazzi" To: gentoo-commits@lists.gentoo.org Content-Transfer-Encoding: 8bit Content-type: text/plain; charset=UTF-8 Reply-To: gentoo-dev@lists.gentoo.org, "Alice Ferrazzi" Message-ID: <1514567504.992b1f11ac9ee508ceb2448f133ed9f256a14ee9.alicef@gentoo> Subject: [gentoo-commits] proj/linux-patches:4.14 commit in: / X-VCS-Repository: proj/linux-patches X-VCS-Files: 0000_README 1009_linux-4.14.10.patch X-VCS-Directories: / X-VCS-Committer: alicef X-VCS-Committer-Name: Alice Ferrazzi X-VCS-Revision: 992b1f11ac9ee508ceb2448f133ed9f256a14ee9 X-VCS-Branch: 4.14 Date: Fri, 29 Dec 2017 17:18:41 +0000 (UTC) Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Gentoo Linux mail X-BeenThere: gentoo-commits@lists.gentoo.org X-Archives-Salt: a184d8fe-9d43-4944-af2e-b53a7ef5011e X-Archives-Hash: 6f485caf6849f5e883405f54d9054ac5 commit: 992b1f11ac9ee508ceb2448f133ed9f256a14ee9 Author: Alice Ferrazzi gentoo org> AuthorDate: Fri Dec 29 17:11:44 2017 +0000 Commit: Alice Ferrazzi gentoo org> CommitDate: Fri Dec 29 17:11:44 2017 +0000 URL: https://gitweb.gentoo.org/proj/linux-patches.git/commit/?id=992b1f11 linux kernel 4.14.10 0000_README | 4 + 1009_linux-4.14.10.patch | 9685 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 9689 insertions(+) diff --git a/0000_README b/0000_README index a19ca77..e43b606 100644 --- a/0000_README +++ b/0000_README @@ -79,6 +79,10 @@ Patch: 1008_linux-4.14.9.patch From: http://www.kernel.org Desc: Linux 4.14.9 +Patch: 1009_linux-4.14.10.patch +From: http://www.kernel.org +Desc: Linux 4.14.10 + Patch: 1500_XATTR_USER_PREFIX.patch From: https://bugs.gentoo.org/show_bug.cgi?id=470644 Desc: Support for namespace user.pax.* on tmpfs. diff --git a/1009_linux-4.14.10.patch b/1009_linux-4.14.10.patch new file mode 100644 index 0000000..85a0a1f --- /dev/null +++ b/1009_linux-4.14.10.patch @@ -0,0 +1,9685 @@ +diff --git a/Documentation/x86/x86_64/mm.txt b/Documentation/x86/x86_64/mm.txt +index 3448e675b462..51101708a03a 100644 +--- a/Documentation/x86/x86_64/mm.txt ++++ b/Documentation/x86/x86_64/mm.txt +@@ -1,6 +1,4 @@ + +- +- + Virtual memory map with 4 level page tables: + + 0000000000000000 - 00007fffffffffff (=47 bits) user space, different per mm +@@ -14,13 +12,15 @@ ffffea0000000000 - ffffeaffffffffff (=40 bits) virtual memory map (1TB) + ... unused hole ... + ffffec0000000000 - fffffbffffffffff (=44 bits) kasan shadow memory (16TB) + ... unused hole ... ++fffffe8000000000 - fffffeffffffffff (=39 bits) cpu_entry_area mapping + ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks + ... unused hole ... + ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space + ... unused hole ... + ffffffff80000000 - ffffffff9fffffff (=512 MB) kernel text mapping, from phys 0 +-ffffffffa0000000 - ffffffffff5fffff (=1526 MB) module mapping space (variable) +-ffffffffff600000 - ffffffffffdfffff (=8 MB) vsyscalls ++ffffffffa0000000 - [fixmap start] (~1526 MB) module mapping space (variable) ++[fixmap start] - ffffffffff5fffff kernel-internal fixmap range ++ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI + ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole + + Virtual memory map with 5 level page tables: +@@ -36,19 +36,22 @@ ffd4000000000000 - ffd5ffffffffffff (=49 bits) virtual memory map (512TB) + ... unused hole ... + ffdf000000000000 - fffffc0000000000 (=53 bits) kasan shadow memory (8PB) + ... unused hole ... ++fffffe8000000000 - fffffeffffffffff (=39 bits) cpu_entry_area mapping + ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks + ... unused hole ... + ffffffef00000000 - fffffffeffffffff (=64 GB) EFI region mapping space + ... unused hole ... + ffffffff80000000 - ffffffff9fffffff (=512 MB) kernel text mapping, from phys 0 +-ffffffffa0000000 - ffffffffff5fffff (=1526 MB) module mapping space +-ffffffffff600000 - ffffffffffdfffff (=8 MB) vsyscalls ++ffffffffa0000000 - [fixmap start] (~1526 MB) module mapping space ++[fixmap start] - ffffffffff5fffff kernel-internal fixmap range ++ffffffffff600000 - ffffffffff600fff (=4 kB) legacy vsyscall ABI + ffffffffffe00000 - ffffffffffffffff (=2 MB) unused hole + + Architecture defines a 64-bit virtual address. Implementations can support + less. Currently supported are 48- and 57-bit virtual addresses. Bits 63 +-through to the most-significant implemented bit are set to either all ones +-or all zero. This causes hole between user space and kernel addresses. ++through to the most-significant implemented bit are sign extended. ++This causes hole between user space and kernel addresses if you interpret them ++as unsigned. + + The direct mapping covers all memory in the system up to the highest + memory address (this means in some cases it can also include PCI memory +@@ -58,9 +61,6 @@ vmalloc space is lazily synchronized into the different PML4/PML5 pages of + the processes using the page fault handler, with init_top_pgt as + reference. + +-Current X86-64 implementations support up to 46 bits of address space (64 TB), +-which is our current limit. This expands into MBZ space in the page tables. +- + We map EFI runtime services in the 'efi_pgd' PGD in a 64Gb large virtual + memory window (this size is arbitrary, it can be raised later if needed). + The mappings are not part of any other kernel PGD and are only available +@@ -72,5 +72,3 @@ following fixmap section. + Note that if CONFIG_RANDOMIZE_MEMORY is enabled, the direct mapping of all + physical memory, vmalloc/ioremap space and virtual memory map are randomized. + Their order is preserved but their base will be offset early at boot time. +- +--Andi Kleen, Jul 2004 +diff --git a/Makefile b/Makefile +index ed2132c6d286..9edfb78836a9 100644 +--- a/Makefile ++++ b/Makefile +@@ -1,7 +1,7 @@ + # SPDX-License-Identifier: GPL-2.0 + VERSION = 4 + PATCHLEVEL = 14 +-SUBLEVEL = 9 ++SUBLEVEL = 10 + EXTRAVERSION = + NAME = Petit Gorille + +diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c +index f5154ed3da6c..2add22699764 100644 +--- a/arch/arm64/kvm/hyp/debug-sr.c ++++ b/arch/arm64/kvm/hyp/debug-sr.c +@@ -84,6 +84,9 @@ static void __hyp_text __debug_save_spe_nvhe(u64 *pmscr_el1) + { + u64 reg; + ++ /* Clear pmscr in case of early return */ ++ *pmscr_el1 = 0; ++ + /* SPE present on this CPU? */ + if (!cpuid_feature_extract_unsigned_field(read_sysreg(id_aa64dfr0_el1), + ID_AA64DFR0_PMSVER_SHIFT)) +diff --git a/arch/parisc/boot/compressed/misc.c b/arch/parisc/boot/compressed/misc.c +index 9345b44b86f0..f57118e1f6b4 100644 +--- a/arch/parisc/boot/compressed/misc.c ++++ b/arch/parisc/boot/compressed/misc.c +@@ -123,8 +123,8 @@ int puts(const char *s) + while ((nuline = strchr(s, '\n')) != NULL) { + if (nuline != s) + pdc_iodc_print(s, nuline - s); +- pdc_iodc_print("\r\n", 2); +- s = nuline + 1; ++ pdc_iodc_print("\r\n", 2); ++ s = nuline + 1; + } + if (*s != '\0') + pdc_iodc_print(s, strlen(s)); +diff --git a/arch/parisc/kernel/entry.S b/arch/parisc/kernel/entry.S +index a4fd296c958e..f3cecf5117cf 100644 +--- a/arch/parisc/kernel/entry.S ++++ b/arch/parisc/kernel/entry.S +@@ -878,9 +878,6 @@ ENTRY_CFI(syscall_exit_rfi) + STREG %r19,PT_SR7(%r16) + + intr_return: +- /* NOTE: Need to enable interrupts incase we schedule. */ +- ssm PSW_SM_I, %r0 +- + /* check for reschedule */ + mfctl %cr30,%r1 + LDREG TI_FLAGS(%r1),%r19 /* sched.h: TIF_NEED_RESCHED */ +@@ -907,6 +904,11 @@ intr_check_sig: + LDREG PT_IASQ1(%r16), %r20 + cmpib,COND(=),n 0,%r20,intr_restore /* backward */ + ++ /* NOTE: We need to enable interrupts if we have to deliver ++ * signals. We used to do this earlier but it caused kernel ++ * stack overflows. */ ++ ssm PSW_SM_I, %r0 ++ + copy %r0, %r25 /* long in_syscall = 0 */ + #ifdef CONFIG_64BIT + ldo -16(%r30),%r29 /* Reference param save area */ +@@ -958,6 +960,10 @@ intr_do_resched: + cmpib,COND(=) 0, %r20, intr_do_preempt + nop + ++ /* NOTE: We need to enable interrupts if we schedule. We used ++ * to do this earlier but it caused kernel stack overflows. */ ++ ssm PSW_SM_I, %r0 ++ + #ifdef CONFIG_64BIT + ldo -16(%r30),%r29 /* Reference param save area */ + #endif +diff --git a/arch/parisc/kernel/hpmc.S b/arch/parisc/kernel/hpmc.S +index e3a8e5e4d5de..8d072c44f300 100644 +--- a/arch/parisc/kernel/hpmc.S ++++ b/arch/parisc/kernel/hpmc.S +@@ -305,6 +305,7 @@ ENDPROC_CFI(os_hpmc) + + + __INITRODATA ++ .align 4 + .export os_hpmc_size + os_hpmc_size: + .word .os_hpmc_end-.os_hpmc +diff --git a/arch/powerpc/include/asm/mmu_context.h b/arch/powerpc/include/asm/mmu_context.h +index 492d8140a395..44fdf4786638 100644 +--- a/arch/powerpc/include/asm/mmu_context.h ++++ b/arch/powerpc/include/asm/mmu_context.h +@@ -114,9 +114,10 @@ static inline void enter_lazy_tlb(struct mm_struct *mm, + #endif + } + +-static inline void arch_dup_mmap(struct mm_struct *oldmm, +- struct mm_struct *mm) ++static inline int arch_dup_mmap(struct mm_struct *oldmm, ++ struct mm_struct *mm) + { ++ return 0; + } + + static inline void arch_exit_mmap(struct mm_struct *mm) +diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c +index bf457843e032..0d750d274c4e 100644 +--- a/arch/powerpc/kvm/book3s_xive.c ++++ b/arch/powerpc/kvm/book3s_xive.c +@@ -725,7 +725,8 @@ u64 kvmppc_xive_get_icp(struct kvm_vcpu *vcpu) + + /* Return the per-cpu state for state saving/migration */ + return (u64)xc->cppr << KVM_REG_PPC_ICP_CPPR_SHIFT | +- (u64)xc->mfrr << KVM_REG_PPC_ICP_MFRR_SHIFT; ++ (u64)xc->mfrr << KVM_REG_PPC_ICP_MFRR_SHIFT | ++ (u64)0xff << KVM_REG_PPC_ICP_PPRI_SHIFT; + } + + int kvmppc_xive_set_icp(struct kvm_vcpu *vcpu, u64 icpval) +@@ -1558,7 +1559,7 @@ static int xive_set_source(struct kvmppc_xive *xive, long irq, u64 addr) + + /* + * Restore P and Q. If the interrupt was pending, we +- * force both P and Q, which will trigger a resend. ++ * force Q and !P, which will trigger a resend. + * + * That means that a guest that had both an interrupt + * pending (queued) and Q set will restore with only +@@ -1566,7 +1567,7 @@ static int xive_set_source(struct kvmppc_xive *xive, long irq, u64 addr) + * is perfectly fine as coalescing interrupts that haven't + * been presented yet is always allowed. + */ +- if (val & KVM_XICS_PRESENTED || val & KVM_XICS_PENDING) ++ if (val & KVM_XICS_PRESENTED && !(val & KVM_XICS_PENDING)) + state->old_p = true; + if (val & KVM_XICS_QUEUED || val & KVM_XICS_PENDING) + state->old_q = true; +diff --git a/arch/powerpc/perf/core-book3s.c b/arch/powerpc/perf/core-book3s.c +index 9e3da168d54c..b4209a68b85d 100644 +--- a/arch/powerpc/perf/core-book3s.c ++++ b/arch/powerpc/perf/core-book3s.c +@@ -410,8 +410,12 @@ static __u64 power_pmu_bhrb_to(u64 addr) + int ret; + __u64 target; + +- if (is_kernel_addr(addr)) +- return branch_target((unsigned int *)addr); ++ if (is_kernel_addr(addr)) { ++ if (probe_kernel_read(&instr, (void *)addr, sizeof(instr))) ++ return 0; ++ ++ return branch_target(&instr); ++ } + + /* Userspace: need copy instruction here then translate it */ + pagefault_disable(); +diff --git a/arch/um/include/asm/mmu_context.h b/arch/um/include/asm/mmu_context.h +index b668e351fd6c..fca34b2177e2 100644 +--- a/arch/um/include/asm/mmu_context.h ++++ b/arch/um/include/asm/mmu_context.h +@@ -15,9 +15,10 @@ extern void uml_setup_stubs(struct mm_struct *mm); + /* + * Needed since we do not use the asm-generic/mm_hooks.h: + */ +-static inline void arch_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm) ++static inline int arch_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm) + { + uml_setup_stubs(mm); ++ return 0; + } + extern void arch_exit_mmap(struct mm_struct *mm); + static inline void arch_unmap(struct mm_struct *mm, +diff --git a/arch/unicore32/include/asm/mmu_context.h b/arch/unicore32/include/asm/mmu_context.h +index 59b06b48f27d..5c205a9cb5a6 100644 +--- a/arch/unicore32/include/asm/mmu_context.h ++++ b/arch/unicore32/include/asm/mmu_context.h +@@ -81,9 +81,10 @@ do { \ + } \ + } while (0) + +-static inline void arch_dup_mmap(struct mm_struct *oldmm, +- struct mm_struct *mm) ++static inline int arch_dup_mmap(struct mm_struct *oldmm, ++ struct mm_struct *mm) + { ++ return 0; + } + + static inline void arch_unmap(struct mm_struct *mm, +diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig +index 48646160eb83..592c974d4558 100644 +--- a/arch/x86/Kconfig ++++ b/arch/x86/Kconfig +@@ -925,7 +925,8 @@ config MAXSMP + config NR_CPUS + int "Maximum number of CPUs" if SMP && !MAXSMP + range 2 8 if SMP && X86_32 && !X86_BIGSMP +- range 2 512 if SMP && !MAXSMP && !CPUMASK_OFFSTACK ++ range 2 64 if SMP && X86_32 && X86_BIGSMP ++ range 2 512 if SMP && !MAXSMP && !CPUMASK_OFFSTACK && X86_64 + range 2 8192 if SMP && !MAXSMP && CPUMASK_OFFSTACK && X86_64 + default "1" if !SMP + default "8192" if MAXSMP +diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S +index bd8b57a5c874..ace8f321a5a1 100644 +--- a/arch/x86/entry/entry_32.S ++++ b/arch/x86/entry/entry_32.S +@@ -942,9 +942,9 @@ ENTRY(debug) + + /* Are we currently on the SYSENTER stack? */ + movl PER_CPU_VAR(cpu_entry_area), %ecx +- addl $CPU_ENTRY_AREA_SYSENTER_stack + SIZEOF_SYSENTER_stack, %ecx +- subl %eax, %ecx /* ecx = (end of SYSENTER_stack) - esp */ +- cmpl $SIZEOF_SYSENTER_stack, %ecx ++ addl $CPU_ENTRY_AREA_entry_stack + SIZEOF_entry_stack, %ecx ++ subl %eax, %ecx /* ecx = (end of entry_stack) - esp */ ++ cmpl $SIZEOF_entry_stack, %ecx + jb .Ldebug_from_sysenter_stack + + TRACE_IRQS_OFF +@@ -986,9 +986,9 @@ ENTRY(nmi) + + /* Are we currently on the SYSENTER stack? */ + movl PER_CPU_VAR(cpu_entry_area), %ecx +- addl $CPU_ENTRY_AREA_SYSENTER_stack + SIZEOF_SYSENTER_stack, %ecx +- subl %eax, %ecx /* ecx = (end of SYSENTER_stack) - esp */ +- cmpl $SIZEOF_SYSENTER_stack, %ecx ++ addl $CPU_ENTRY_AREA_entry_stack + SIZEOF_entry_stack, %ecx ++ subl %eax, %ecx /* ecx = (end of entry_stack) - esp */ ++ cmpl $SIZEOF_entry_stack, %ecx + jb .Lnmi_from_sysenter_stack + + /* Not on SYSENTER stack. */ +diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S +index 6abe3fcaece9..22c891c3b78d 100644 +--- a/arch/x86/entry/entry_64.S ++++ b/arch/x86/entry/entry_64.S +@@ -154,8 +154,8 @@ END(native_usergs_sysret64) + _entry_trampoline - CPU_ENTRY_AREA_entry_trampoline(%rip) + + /* The top word of the SYSENTER stack is hot and is usable as scratch space. */ +-#define RSP_SCRATCH CPU_ENTRY_AREA_SYSENTER_stack + \ +- SIZEOF_SYSENTER_stack - 8 + CPU_ENTRY_AREA ++#define RSP_SCRATCH CPU_ENTRY_AREA_entry_stack + \ ++ SIZEOF_entry_stack - 8 + CPU_ENTRY_AREA + + ENTRY(entry_SYSCALL_64_trampoline) + UNWIND_HINT_EMPTY +diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscall/vsyscall_64.c +index f279ba2643dc..1faf40f2dda9 100644 +--- a/arch/x86/entry/vsyscall/vsyscall_64.c ++++ b/arch/x86/entry/vsyscall/vsyscall_64.c +@@ -37,6 +37,7 @@ + #include + #include + #include ++#include + + #define CREATE_TRACE_POINTS + #include "vsyscall_trace.h" +@@ -138,6 +139,10 @@ bool emulate_vsyscall(struct pt_regs *regs, unsigned long address) + + WARN_ON_ONCE(address != regs->ip); + ++ /* This should be unreachable in NATIVE mode. */ ++ if (WARN_ON(vsyscall_mode == NATIVE)) ++ return false; ++ + if (vsyscall_mode == NONE) { + warn_bad_vsyscall(KERN_INFO, regs, + "vsyscall attempted with vsyscall=none"); +@@ -329,16 +334,47 @@ int in_gate_area_no_mm(unsigned long addr) + return vsyscall_mode != NONE && (addr & PAGE_MASK) == VSYSCALL_ADDR; + } + ++/* ++ * The VSYSCALL page is the only user-accessible page in the kernel address ++ * range. Normally, the kernel page tables can have _PAGE_USER clear, but ++ * the tables covering VSYSCALL_ADDR need _PAGE_USER set if vsyscalls ++ * are enabled. ++ * ++ * Some day we may create a "minimal" vsyscall mode in which we emulate ++ * vsyscalls but leave the page not present. If so, we skip calling ++ * this. ++ */ ++static void __init set_vsyscall_pgtable_user_bits(void) ++{ ++ pgd_t *pgd; ++ p4d_t *p4d; ++ pud_t *pud; ++ pmd_t *pmd; ++ ++ pgd = pgd_offset_k(VSYSCALL_ADDR); ++ set_pgd(pgd, __pgd(pgd_val(*pgd) | _PAGE_USER)); ++ p4d = p4d_offset(pgd, VSYSCALL_ADDR); ++#if CONFIG_PGTABLE_LEVELS >= 5 ++ p4d->p4d |= _PAGE_USER; ++#endif ++ pud = pud_offset(p4d, VSYSCALL_ADDR); ++ set_pud(pud, __pud(pud_val(*pud) | _PAGE_USER)); ++ pmd = pmd_offset(pud, VSYSCALL_ADDR); ++ set_pmd(pmd, __pmd(pmd_val(*pmd) | _PAGE_USER)); ++} ++ + void __init map_vsyscall(void) + { + extern char __vsyscall_page; + unsigned long physaddr_vsyscall = __pa_symbol(&__vsyscall_page); + +- if (vsyscall_mode != NONE) ++ if (vsyscall_mode != NONE) { + __set_fixmap(VSYSCALL_PAGE, physaddr_vsyscall, + vsyscall_mode == NATIVE + ? PAGE_KERNEL_VSYSCALL + : PAGE_KERNEL_VVAR); ++ set_vsyscall_pgtable_user_bits(); ++ } + + BUILD_BUG_ON((unsigned long)__fix_to_virt(VSYSCALL_PAGE) != + (unsigned long)VSYSCALL_ADDR); +diff --git a/arch/x86/include/asm/cpu_entry_area.h b/arch/x86/include/asm/cpu_entry_area.h +new file mode 100644 +index 000000000000..2fbc69a0916e +--- /dev/null ++++ b/arch/x86/include/asm/cpu_entry_area.h +@@ -0,0 +1,68 @@ ++// SPDX-License-Identifier: GPL-2.0 ++ ++#ifndef _ASM_X86_CPU_ENTRY_AREA_H ++#define _ASM_X86_CPU_ENTRY_AREA_H ++ ++#include ++#include ++ ++/* ++ * cpu_entry_area is a percpu region that contains things needed by the CPU ++ * and early entry/exit code. Real types aren't used for all fields here ++ * to avoid circular header dependencies. ++ * ++ * Every field is a virtual alias of some other allocated backing store. ++ * There is no direct allocation of a struct cpu_entry_area. ++ */ ++struct cpu_entry_area { ++ char gdt[PAGE_SIZE]; ++ ++ /* ++ * The GDT is just below entry_stack and thus serves (on x86_64) as ++ * a a read-only guard page. ++ */ ++ struct entry_stack_page entry_stack_page; ++ ++ /* ++ * On x86_64, the TSS is mapped RO. On x86_32, it's mapped RW because ++ * we need task switches to work, and task switches write to the TSS. ++ */ ++ struct tss_struct tss; ++ ++ char entry_trampoline[PAGE_SIZE]; ++ ++#ifdef CONFIG_X86_64 ++ /* ++ * Exception stacks used for IST entries. ++ * ++ * In the future, this should have a separate slot for each stack ++ * with guard pages between them. ++ */ ++ char exception_stacks[(N_EXCEPTION_STACKS - 1) * EXCEPTION_STKSZ + DEBUG_STKSZ]; ++#endif ++}; ++ ++#define CPU_ENTRY_AREA_SIZE (sizeof(struct cpu_entry_area)) ++#define CPU_ENTRY_AREA_TOT_SIZE (CPU_ENTRY_AREA_SIZE * NR_CPUS) ++ ++DECLARE_PER_CPU(struct cpu_entry_area *, cpu_entry_area); ++ ++extern void setup_cpu_entry_areas(void); ++extern void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags); ++ ++#define CPU_ENTRY_AREA_RO_IDT CPU_ENTRY_AREA_BASE ++#define CPU_ENTRY_AREA_PER_CPU (CPU_ENTRY_AREA_RO_IDT + PAGE_SIZE) ++ ++#define CPU_ENTRY_AREA_RO_IDT_VADDR ((void *)CPU_ENTRY_AREA_RO_IDT) ++ ++#define CPU_ENTRY_AREA_MAP_SIZE \ ++ (CPU_ENTRY_AREA_PER_CPU + CPU_ENTRY_AREA_TOT_SIZE - CPU_ENTRY_AREA_BASE) ++ ++extern struct cpu_entry_area *get_cpu_entry_area(int cpu); ++ ++static inline struct entry_stack *cpu_entry_stack(int cpu) ++{ ++ return &get_cpu_entry_area(cpu)->entry_stack_page.stack; ++} ++ ++#endif +diff --git a/arch/x86/include/asm/desc.h b/arch/x86/include/asm/desc.h +index 2ace1f90d138..bc359dd2f7f6 100644 +--- a/arch/x86/include/asm/desc.h ++++ b/arch/x86/include/asm/desc.h +@@ -7,6 +7,7 @@ + #include + #include + #include ++#include + + #include + #include +diff --git a/arch/x86/include/asm/espfix.h b/arch/x86/include/asm/espfix.h +index 0211029076ea..6777480d8a42 100644 +--- a/arch/x86/include/asm/espfix.h ++++ b/arch/x86/include/asm/espfix.h +@@ -2,7 +2,7 @@ + #ifndef _ASM_X86_ESPFIX_H + #define _ASM_X86_ESPFIX_H + +-#ifdef CONFIG_X86_64 ++#ifdef CONFIG_X86_ESPFIX64 + + #include + +@@ -11,7 +11,8 @@ DECLARE_PER_CPU_READ_MOSTLY(unsigned long, espfix_waddr); + + extern void init_espfix_bsp(void); + extern void init_espfix_ap(int cpu); +- +-#endif /* CONFIG_X86_64 */ ++#else ++static inline void init_espfix_ap(int cpu) { } ++#endif + + #endif /* _ASM_X86_ESPFIX_H */ +diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h +index 94fc4fa14127..64c4a30e0d39 100644 +--- a/arch/x86/include/asm/fixmap.h ++++ b/arch/x86/include/asm/fixmap.h +@@ -44,46 +44,6 @@ extern unsigned long __FIXADDR_TOP; + PAGE_SIZE) + #endif + +-/* +- * cpu_entry_area is a percpu region in the fixmap that contains things +- * needed by the CPU and early entry/exit code. Real types aren't used +- * for all fields here to avoid circular header dependencies. +- * +- * Every field is a virtual alias of some other allocated backing store. +- * There is no direct allocation of a struct cpu_entry_area. +- */ +-struct cpu_entry_area { +- char gdt[PAGE_SIZE]; +- +- /* +- * The GDT is just below SYSENTER_stack and thus serves (on x86_64) as +- * a a read-only guard page. +- */ +- struct SYSENTER_stack_page SYSENTER_stack_page; +- +- /* +- * On x86_64, the TSS is mapped RO. On x86_32, it's mapped RW because +- * we need task switches to work, and task switches write to the TSS. +- */ +- struct tss_struct tss; +- +- char entry_trampoline[PAGE_SIZE]; +- +-#ifdef CONFIG_X86_64 +- /* +- * Exception stacks used for IST entries. +- * +- * In the future, this should have a separate slot for each stack +- * with guard pages between them. +- */ +- char exception_stacks[(N_EXCEPTION_STACKS - 1) * EXCEPTION_STKSZ + DEBUG_STKSZ]; +-#endif +-}; +- +-#define CPU_ENTRY_AREA_PAGES (sizeof(struct cpu_entry_area) / PAGE_SIZE) +- +-extern void setup_cpu_entry_areas(void); +- + /* + * Here we define all the compile-time 'special' virtual + * addresses. The point is to have a constant address at +@@ -123,7 +83,6 @@ enum fixed_addresses { + FIX_IO_APIC_BASE_0, + FIX_IO_APIC_BASE_END = FIX_IO_APIC_BASE_0 + MAX_IO_APICS - 1, + #endif +- FIX_RO_IDT, /* Virtual mapping for read-only IDT */ + #ifdef CONFIG_X86_32 + FIX_KMAP_BEGIN, /* reserved pte's for temporary kernel mappings */ + FIX_KMAP_END = FIX_KMAP_BEGIN+(KM_TYPE_NR*NR_CPUS)-1, +@@ -139,9 +98,6 @@ enum fixed_addresses { + #ifdef CONFIG_X86_INTEL_MID + FIX_LNW_VRTC, + #endif +- /* Fixmap entries to remap the GDTs, one per processor. */ +- FIX_CPU_ENTRY_AREA_TOP, +- FIX_CPU_ENTRY_AREA_BOTTOM = FIX_CPU_ENTRY_AREA_TOP + (CPU_ENTRY_AREA_PAGES * NR_CPUS) - 1, + + #ifdef CONFIG_ACPI_APEI_GHES + /* Used for GHES mapping from assorted contexts */ +@@ -182,7 +138,7 @@ enum fixed_addresses { + extern void reserve_top_address(unsigned long reserve); + + #define FIXADDR_SIZE (__end_of_permanent_fixed_addresses << PAGE_SHIFT) +-#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) ++#define FIXADDR_START (FIXADDR_TOP - FIXADDR_SIZE) + + extern int fixmaps_set; + +@@ -230,30 +186,5 @@ void __init *early_memremap_decrypted_wp(resource_size_t phys_addr, + void __early_set_fixmap(enum fixed_addresses idx, + phys_addr_t phys, pgprot_t flags); + +-static inline unsigned int __get_cpu_entry_area_page_index(int cpu, int page) +-{ +- BUILD_BUG_ON(sizeof(struct cpu_entry_area) % PAGE_SIZE != 0); +- +- return FIX_CPU_ENTRY_AREA_BOTTOM - cpu*CPU_ENTRY_AREA_PAGES - page; +-} +- +-#define __get_cpu_entry_area_offset_index(cpu, offset) ({ \ +- BUILD_BUG_ON(offset % PAGE_SIZE != 0); \ +- __get_cpu_entry_area_page_index(cpu, offset / PAGE_SIZE); \ +- }) +- +-#define get_cpu_entry_area_index(cpu, field) \ +- __get_cpu_entry_area_offset_index((cpu), offsetof(struct cpu_entry_area, field)) +- +-static inline struct cpu_entry_area *get_cpu_entry_area(int cpu) +-{ +- return (struct cpu_entry_area *)__fix_to_virt(__get_cpu_entry_area_page_index(cpu, 0)); +-} +- +-static inline struct SYSENTER_stack *cpu_SYSENTER_stack(int cpu) +-{ +- return &get_cpu_entry_area(cpu)->SYSENTER_stack_page.stack; +-} +- + #endif /* !__ASSEMBLY__ */ + #endif /* _ASM_X86_FIXMAP_H */ +diff --git a/arch/x86/include/asm/inat.h b/arch/x86/include/asm/inat.h +index 02aff0867211..1c78580e58be 100644 +--- a/arch/x86/include/asm/inat.h ++++ b/arch/x86/include/asm/inat.h +@@ -97,6 +97,16 @@ + #define INAT_MAKE_GROUP(grp) ((grp << INAT_GRP_OFFS) | INAT_MODRM) + #define INAT_MAKE_IMM(imm) (imm << INAT_IMM_OFFS) + ++/* Identifiers for segment registers */ ++#define INAT_SEG_REG_IGNORE 0 ++#define INAT_SEG_REG_DEFAULT 1 ++#define INAT_SEG_REG_CS 2 ++#define INAT_SEG_REG_SS 3 ++#define INAT_SEG_REG_DS 4 ++#define INAT_SEG_REG_ES 5 ++#define INAT_SEG_REG_FS 6 ++#define INAT_SEG_REG_GS 7 ++ + /* Attribute search APIs */ + extern insn_attr_t inat_get_opcode_attribute(insn_byte_t opcode); + extern int inat_get_last_prefix_id(insn_byte_t last_pfx); +diff --git a/arch/x86/include/asm/invpcid.h b/arch/x86/include/asm/invpcid.h +new file mode 100644 +index 000000000000..989cfa86de85 +--- /dev/null ++++ b/arch/x86/include/asm/invpcid.h +@@ -0,0 +1,53 @@ ++/* SPDX-License-Identifier: GPL-2.0 */ ++#ifndef _ASM_X86_INVPCID ++#define _ASM_X86_INVPCID ++ ++static inline void __invpcid(unsigned long pcid, unsigned long addr, ++ unsigned long type) ++{ ++ struct { u64 d[2]; } desc = { { pcid, addr } }; ++ ++ /* ++ * The memory clobber is because the whole point is to invalidate ++ * stale TLB entries and, especially if we're flushing global ++ * mappings, we don't want the compiler to reorder any subsequent ++ * memory accesses before the TLB flush. ++ * ++ * The hex opcode is invpcid (%ecx), %eax in 32-bit mode and ++ * invpcid (%rcx), %rax in long mode. ++ */ ++ asm volatile (".byte 0x66, 0x0f, 0x38, 0x82, 0x01" ++ : : "m" (desc), "a" (type), "c" (&desc) : "memory"); ++} ++ ++#define INVPCID_TYPE_INDIV_ADDR 0 ++#define INVPCID_TYPE_SINGLE_CTXT 1 ++#define INVPCID_TYPE_ALL_INCL_GLOBAL 2 ++#define INVPCID_TYPE_ALL_NON_GLOBAL 3 ++ ++/* Flush all mappings for a given pcid and addr, not including globals. */ ++static inline void invpcid_flush_one(unsigned long pcid, ++ unsigned long addr) ++{ ++ __invpcid(pcid, addr, INVPCID_TYPE_INDIV_ADDR); ++} ++ ++/* Flush all mappings for a given PCID, not including globals. */ ++static inline void invpcid_flush_single_context(unsigned long pcid) ++{ ++ __invpcid(pcid, 0, INVPCID_TYPE_SINGLE_CTXT); ++} ++ ++/* Flush all mappings, including globals, for all PCIDs. */ ++static inline void invpcid_flush_all(void) ++{ ++ __invpcid(0, 0, INVPCID_TYPE_ALL_INCL_GLOBAL); ++} ++ ++/* Flush all mappings for all PCIDs except globals. */ ++static inline void invpcid_flush_all_nonglobals(void) ++{ ++ __invpcid(0, 0, INVPCID_TYPE_ALL_NON_GLOBAL); ++} ++ ++#endif /* _ASM_X86_INVPCID */ +diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h +index 9ea26f167497..5ff3e8af2c20 100644 +--- a/arch/x86/include/asm/mmu.h ++++ b/arch/x86/include/asm/mmu.h +@@ -3,6 +3,7 @@ + #define _ASM_X86_MMU_H + + #include ++#include + #include + #include + +@@ -27,7 +28,8 @@ typedef struct { + atomic64_t tlb_gen; + + #ifdef CONFIG_MODIFY_LDT_SYSCALL +- struct ldt_struct *ldt; ++ struct rw_semaphore ldt_usr_sem; ++ struct ldt_struct *ldt; + #endif + + #ifdef CONFIG_X86_64 +diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h +index 6d16d15d09a0..5ede7cae1d67 100644 +--- a/arch/x86/include/asm/mmu_context.h ++++ b/arch/x86/include/asm/mmu_context.h +@@ -57,11 +57,17 @@ struct ldt_struct { + /* + * Used for LDT copy/destruction. + */ +-int init_new_context_ldt(struct task_struct *tsk, struct mm_struct *mm); ++static inline void init_new_context_ldt(struct mm_struct *mm) ++{ ++ mm->context.ldt = NULL; ++ init_rwsem(&mm->context.ldt_usr_sem); ++} ++int ldt_dup_context(struct mm_struct *oldmm, struct mm_struct *mm); + void destroy_context_ldt(struct mm_struct *mm); + #else /* CONFIG_MODIFY_LDT_SYSCALL */ +-static inline int init_new_context_ldt(struct task_struct *tsk, +- struct mm_struct *mm) ++static inline void init_new_context_ldt(struct mm_struct *mm) { } ++static inline int ldt_dup_context(struct mm_struct *oldmm, ++ struct mm_struct *mm) + { + return 0; + } +@@ -132,18 +138,21 @@ void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk); + static inline int init_new_context(struct task_struct *tsk, + struct mm_struct *mm) + { ++ mutex_init(&mm->context.lock); ++ + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); + atomic64_set(&mm->context.tlb_gen, 0); + +- #ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS ++#ifdef CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS + if (cpu_feature_enabled(X86_FEATURE_OSPKE)) { + /* pkey 0 is the default and always allocated */ + mm->context.pkey_allocation_map = 0x1; + /* -1 means unallocated or invalid */ + mm->context.execute_only_pkey = -1; + } +- #endif +- return init_new_context_ldt(tsk, mm); ++#endif ++ init_new_context_ldt(mm); ++ return 0; + } + static inline void destroy_context(struct mm_struct *mm) + { +@@ -176,10 +185,10 @@ do { \ + } while (0) + #endif + +-static inline void arch_dup_mmap(struct mm_struct *oldmm, +- struct mm_struct *mm) ++static inline int arch_dup_mmap(struct mm_struct *oldmm, struct mm_struct *mm) + { + paravirt_arch_dup_mmap(oldmm, mm); ++ return ldt_dup_context(oldmm, mm); + } + + static inline void arch_exit_mmap(struct mm_struct *mm) +@@ -281,33 +290,6 @@ static inline bool arch_vma_access_permitted(struct vm_area_struct *vma, + return __pkru_allows_pkey(vma_pkey(vma), write); + } + +-/* +- * If PCID is on, ASID-aware code paths put the ASID+1 into the PCID +- * bits. This serves two purposes. It prevents a nasty situation in +- * which PCID-unaware code saves CR3, loads some other value (with PCID +- * == 0), and then restores CR3, thus corrupting the TLB for ASID 0 if +- * the saved ASID was nonzero. It also means that any bugs involving +- * loading a PCID-enabled CR3 with CR4.PCIDE off will trigger +- * deterministically. +- */ +- +-static inline unsigned long build_cr3(struct mm_struct *mm, u16 asid) +-{ +- if (static_cpu_has(X86_FEATURE_PCID)) { +- VM_WARN_ON_ONCE(asid > 4094); +- return __sme_pa(mm->pgd) | (asid + 1); +- } else { +- VM_WARN_ON_ONCE(asid != 0); +- return __sme_pa(mm->pgd); +- } +-} +- +-static inline unsigned long build_cr3_noflush(struct mm_struct *mm, u16 asid) +-{ +- VM_WARN_ON_ONCE(asid > 4094); +- return __sme_pa(mm->pgd) | (asid + 1) | CR3_NOFLUSH; +-} +- + /* + * This can be used from process context to figure out what the value of + * CR3 is without needing to do a (slow) __read_cr3(). +@@ -317,7 +299,7 @@ static inline unsigned long build_cr3_noflush(struct mm_struct *mm, u16 asid) + */ + static inline unsigned long __get_current_cr3_fast(void) + { +- unsigned long cr3 = build_cr3(this_cpu_read(cpu_tlbstate.loaded_mm), ++ unsigned long cr3 = build_cr3(this_cpu_read(cpu_tlbstate.loaded_mm)->pgd, + this_cpu_read(cpu_tlbstate.loaded_mm_asid)); + + /* For now, be very restrictive about when this can be called. */ +diff --git a/arch/x86/include/asm/pgtable_32_types.h b/arch/x86/include/asm/pgtable_32_types.h +index f2ca9b28fd68..ce245b0cdfca 100644 +--- a/arch/x86/include/asm/pgtable_32_types.h ++++ b/arch/x86/include/asm/pgtable_32_types.h +@@ -38,13 +38,22 @@ extern bool __vmalloc_start_set; /* set once high_memory is set */ + #define LAST_PKMAP 1024 + #endif + +-#define PKMAP_BASE ((FIXADDR_START - PAGE_SIZE * (LAST_PKMAP + 1)) \ +- & PMD_MASK) ++/* ++ * Define this here and validate with BUILD_BUG_ON() in pgtable_32.c ++ * to avoid include recursion hell ++ */ ++#define CPU_ENTRY_AREA_PAGES (NR_CPUS * 40) ++ ++#define CPU_ENTRY_AREA_BASE \ ++ ((FIXADDR_START - PAGE_SIZE * (CPU_ENTRY_AREA_PAGES + 1)) & PMD_MASK) ++ ++#define PKMAP_BASE \ ++ ((CPU_ENTRY_AREA_BASE - PAGE_SIZE) & PMD_MASK) + + #ifdef CONFIG_HIGHMEM + # define VMALLOC_END (PKMAP_BASE - 2 * PAGE_SIZE) + #else +-# define VMALLOC_END (FIXADDR_START - 2 * PAGE_SIZE) ++# define VMALLOC_END (CPU_ENTRY_AREA_BASE - 2 * PAGE_SIZE) + #endif + + #define MODULES_VADDR VMALLOC_START +diff --git a/arch/x86/include/asm/pgtable_64_types.h b/arch/x86/include/asm/pgtable_64_types.h +index 6d5f45dcd4a1..3d27831bc58d 100644 +--- a/arch/x86/include/asm/pgtable_64_types.h ++++ b/arch/x86/include/asm/pgtable_64_types.h +@@ -76,32 +76,41 @@ typedef struct { pteval_t pte; } pte_t; + #define PGDIR_MASK (~(PGDIR_SIZE - 1)) + + /* See Documentation/x86/x86_64/mm.txt for a description of the memory map. */ +-#define MAXMEM _AC(__AC(1, UL) << MAX_PHYSMEM_BITS, UL) ++#define MAXMEM _AC(__AC(1, UL) << MAX_PHYSMEM_BITS, UL) ++ + #ifdef CONFIG_X86_5LEVEL +-#define VMALLOC_SIZE_TB _AC(16384, UL) +-#define __VMALLOC_BASE _AC(0xff92000000000000, UL) +-#define __VMEMMAP_BASE _AC(0xffd4000000000000, UL) ++# define VMALLOC_SIZE_TB _AC(16384, UL) ++# define __VMALLOC_BASE _AC(0xff92000000000000, UL) ++# define __VMEMMAP_BASE _AC(0xffd4000000000000, UL) + #else +-#define VMALLOC_SIZE_TB _AC(32, UL) +-#define __VMALLOC_BASE _AC(0xffffc90000000000, UL) +-#define __VMEMMAP_BASE _AC(0xffffea0000000000, UL) ++# define VMALLOC_SIZE_TB _AC(32, UL) ++# define __VMALLOC_BASE _AC(0xffffc90000000000, UL) ++# define __VMEMMAP_BASE _AC(0xffffea0000000000, UL) + #endif ++ + #ifdef CONFIG_RANDOMIZE_MEMORY +-#define VMALLOC_START vmalloc_base +-#define VMEMMAP_START vmemmap_base ++# define VMALLOC_START vmalloc_base ++# define VMEMMAP_START vmemmap_base + #else +-#define VMALLOC_START __VMALLOC_BASE +-#define VMEMMAP_START __VMEMMAP_BASE ++# define VMALLOC_START __VMALLOC_BASE ++# define VMEMMAP_START __VMEMMAP_BASE + #endif /* CONFIG_RANDOMIZE_MEMORY */ +-#define VMALLOC_END (VMALLOC_START + _AC((VMALLOC_SIZE_TB << 40) - 1, UL)) +-#define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE) ++ ++#define VMALLOC_END (VMALLOC_START + _AC((VMALLOC_SIZE_TB << 40) - 1, UL)) ++ ++#define MODULES_VADDR (__START_KERNEL_map + KERNEL_IMAGE_SIZE) + /* The module sections ends with the start of the fixmap */ +-#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1) +-#define MODULES_LEN (MODULES_END - MODULES_VADDR) +-#define ESPFIX_PGD_ENTRY _AC(-2, UL) +-#define ESPFIX_BASE_ADDR (ESPFIX_PGD_ENTRY << P4D_SHIFT) +-#define EFI_VA_START ( -4 * (_AC(1, UL) << 30)) +-#define EFI_VA_END (-68 * (_AC(1, UL) << 30)) ++#define MODULES_END __fix_to_virt(__end_of_fixed_addresses + 1) ++#define MODULES_LEN (MODULES_END - MODULES_VADDR) ++ ++#define ESPFIX_PGD_ENTRY _AC(-2, UL) ++#define ESPFIX_BASE_ADDR (ESPFIX_PGD_ENTRY << P4D_SHIFT) ++ ++#define CPU_ENTRY_AREA_PGD _AC(-3, UL) ++#define CPU_ENTRY_AREA_BASE (CPU_ENTRY_AREA_PGD << P4D_SHIFT) ++ ++#define EFI_VA_START ( -4 * (_AC(1, UL) << 30)) ++#define EFI_VA_END (-68 * (_AC(1, UL) << 30)) + + #define EARLY_DYNAMIC_PAGE_TABLES 64 + +diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h +index da943411d3d8..9e482d8b0b97 100644 +--- a/arch/x86/include/asm/processor.h ++++ b/arch/x86/include/asm/processor.h +@@ -336,12 +336,12 @@ struct x86_hw_tss { + #define IO_BITMAP_OFFSET (offsetof(struct tss_struct, io_bitmap) - offsetof(struct tss_struct, x86_tss)) + #define INVALID_IO_BITMAP_OFFSET 0x8000 + +-struct SYSENTER_stack { ++struct entry_stack { + unsigned long words[64]; + }; + +-struct SYSENTER_stack_page { +- struct SYSENTER_stack stack; ++struct entry_stack_page { ++ struct entry_stack stack; + } __aligned(PAGE_SIZE); + + struct tss_struct { +diff --git a/arch/x86/include/asm/stacktrace.h b/arch/x86/include/asm/stacktrace.h +index f8062bfd43a0..f73706878772 100644 +--- a/arch/x86/include/asm/stacktrace.h ++++ b/arch/x86/include/asm/stacktrace.h +@@ -16,7 +16,7 @@ enum stack_type { + STACK_TYPE_TASK, + STACK_TYPE_IRQ, + STACK_TYPE_SOFTIRQ, +- STACK_TYPE_SYSENTER, ++ STACK_TYPE_ENTRY, + STACK_TYPE_EXCEPTION, + STACK_TYPE_EXCEPTION_LAST = STACK_TYPE_EXCEPTION + N_EXCEPTION_STACKS-1, + }; +@@ -29,7 +29,7 @@ struct stack_info { + bool in_task_stack(unsigned long *stack, struct task_struct *task, + struct stack_info *info); + +-bool in_sysenter_stack(unsigned long *stack, struct stack_info *info); ++bool in_entry_stack(unsigned long *stack, struct stack_info *info); + + int get_stack_info(unsigned long *stack, struct task_struct *task, + struct stack_info *info, unsigned long *visit_mask); +diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h +index 509046cfa5ce..171b429f43a2 100644 +--- a/arch/x86/include/asm/tlbflush.h ++++ b/arch/x86/include/asm/tlbflush.h +@@ -9,70 +9,66 @@ + #include + #include + #include ++#include + +-static inline void __invpcid(unsigned long pcid, unsigned long addr, +- unsigned long type) ++static inline u64 inc_mm_tlb_gen(struct mm_struct *mm) + { +- struct { u64 d[2]; } desc = { { pcid, addr } }; +- + /* +- * The memory clobber is because the whole point is to invalidate +- * stale TLB entries and, especially if we're flushing global +- * mappings, we don't want the compiler to reorder any subsequent +- * memory accesses before the TLB flush. +- * +- * The hex opcode is invpcid (%ecx), %eax in 32-bit mode and +- * invpcid (%rcx), %rax in long mode. ++ * Bump the generation count. This also serves as a full barrier ++ * that synchronizes with switch_mm(): callers are required to order ++ * their read of mm_cpumask after their writes to the paging ++ * structures. + */ +- asm volatile (".byte 0x66, 0x0f, 0x38, 0x82, 0x01" +- : : "m" (desc), "a" (type), "c" (&desc) : "memory"); ++ return atomic64_inc_return(&mm->context.tlb_gen); + } + +-#define INVPCID_TYPE_INDIV_ADDR 0 +-#define INVPCID_TYPE_SINGLE_CTXT 1 +-#define INVPCID_TYPE_ALL_INCL_GLOBAL 2 +-#define INVPCID_TYPE_ALL_NON_GLOBAL 3 +- +-/* Flush all mappings for a given pcid and addr, not including globals. */ +-static inline void invpcid_flush_one(unsigned long pcid, +- unsigned long addr) +-{ +- __invpcid(pcid, addr, INVPCID_TYPE_INDIV_ADDR); +-} ++/* There are 12 bits of space for ASIDS in CR3 */ ++#define CR3_HW_ASID_BITS 12 ++/* ++ * When enabled, PAGE_TABLE_ISOLATION consumes a single bit for ++ * user/kernel switches ++ */ ++#define PTI_CONSUMED_ASID_BITS 0 + +-/* Flush all mappings for a given PCID, not including globals. */ +-static inline void invpcid_flush_single_context(unsigned long pcid) +-{ +- __invpcid(pcid, 0, INVPCID_TYPE_SINGLE_CTXT); +-} ++#define CR3_AVAIL_ASID_BITS (CR3_HW_ASID_BITS - PTI_CONSUMED_ASID_BITS) ++/* ++ * ASIDs are zero-based: 0->MAX_AVAIL_ASID are valid. -1 below to account ++ * for them being zero-based. Another -1 is because ASID 0 is reserved for ++ * use by non-PCID-aware users. ++ */ ++#define MAX_ASID_AVAILABLE ((1 << CR3_AVAIL_ASID_BITS) - 2) + +-/* Flush all mappings, including globals, for all PCIDs. */ +-static inline void invpcid_flush_all(void) ++static inline u16 kern_pcid(u16 asid) + { +- __invpcid(0, 0, INVPCID_TYPE_ALL_INCL_GLOBAL); ++ VM_WARN_ON_ONCE(asid > MAX_ASID_AVAILABLE); ++ /* ++ * If PCID is on, ASID-aware code paths put the ASID+1 into the ++ * PCID bits. This serves two purposes. It prevents a nasty ++ * situation in which PCID-unaware code saves CR3, loads some other ++ * value (with PCID == 0), and then restores CR3, thus corrupting ++ * the TLB for ASID 0 if the saved ASID was nonzero. It also means ++ * that any bugs involving loading a PCID-enabled CR3 with ++ * CR4.PCIDE off will trigger deterministically. ++ */ ++ return asid + 1; + } + +-/* Flush all mappings for all PCIDs except globals. */ +-static inline void invpcid_flush_all_nonglobals(void) ++struct pgd_t; ++static inline unsigned long build_cr3(pgd_t *pgd, u16 asid) + { +- __invpcid(0, 0, INVPCID_TYPE_ALL_NON_GLOBAL); ++ if (static_cpu_has(X86_FEATURE_PCID)) { ++ return __sme_pa(pgd) | kern_pcid(asid); ++ } else { ++ VM_WARN_ON_ONCE(asid != 0); ++ return __sme_pa(pgd); ++ } + } + +-static inline u64 inc_mm_tlb_gen(struct mm_struct *mm) ++static inline unsigned long build_cr3_noflush(pgd_t *pgd, u16 asid) + { +- u64 new_tlb_gen; +- +- /* +- * Bump the generation count. This also serves as a full barrier +- * that synchronizes with switch_mm(): callers are required to order +- * their read of mm_cpumask after their writes to the paging +- * structures. +- */ +- smp_mb__before_atomic(); +- new_tlb_gen = atomic64_inc_return(&mm->context.tlb_gen); +- smp_mb__after_atomic(); +- +- return new_tlb_gen; ++ VM_WARN_ON_ONCE(asid > MAX_ASID_AVAILABLE); ++ VM_WARN_ON_ONCE(!this_cpu_has(X86_FEATURE_PCID)); ++ return __sme_pa(pgd) | kern_pcid(asid) | CR3_NOFLUSH; + } + + #ifdef CONFIG_PARAVIRT +@@ -234,6 +230,9 @@ static inline void cr4_set_bits_and_update_boot(unsigned long mask) + + extern void initialize_tlbstate_and_flush(void); + ++/* ++ * flush the entire current user mapping ++ */ + static inline void __native_flush_tlb(void) + { + /* +@@ -246,20 +245,12 @@ static inline void __native_flush_tlb(void) + preempt_enable(); + } + +-static inline void __native_flush_tlb_global_irq_disabled(void) +-{ +- unsigned long cr4; +- +- cr4 = this_cpu_read(cpu_tlbstate.cr4); +- /* clear PGE */ +- native_write_cr4(cr4 & ~X86_CR4_PGE); +- /* write old PGE again and flush TLBs */ +- native_write_cr4(cr4); +-} +- ++/* ++ * flush everything ++ */ + static inline void __native_flush_tlb_global(void) + { +- unsigned long flags; ++ unsigned long cr4, flags; + + if (static_cpu_has(X86_FEATURE_INVPCID)) { + /* +@@ -277,22 +268,36 @@ static inline void __native_flush_tlb_global(void) + */ + raw_local_irq_save(flags); + +- __native_flush_tlb_global_irq_disabled(); ++ cr4 = this_cpu_read(cpu_tlbstate.cr4); ++ /* toggle PGE */ ++ native_write_cr4(cr4 ^ X86_CR4_PGE); ++ /* write old PGE again and flush TLBs */ ++ native_write_cr4(cr4); + + raw_local_irq_restore(flags); + } + ++/* ++ * flush one page in the user mapping ++ */ + static inline void __native_flush_tlb_single(unsigned long addr) + { + asm volatile("invlpg (%0)" ::"r" (addr) : "memory"); + } + ++/* ++ * flush everything ++ */ + static inline void __flush_tlb_all(void) + { +- if (boot_cpu_has(X86_FEATURE_PGE)) ++ if (boot_cpu_has(X86_FEATURE_PGE)) { + __flush_tlb_global(); +- else ++ } else { ++ /* ++ * !PGE -> !PCID (setup_pcid()), thus every flush is total. ++ */ + __flush_tlb(); ++ } + + /* + * Note: if we somehow had PCID but not PGE, then this wouldn't work -- +@@ -303,6 +308,9 @@ static inline void __flush_tlb_all(void) + */ + } + ++/* ++ * flush one page in the kernel mapping ++ */ + static inline void __flush_tlb_one(unsigned long addr) + { + count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE); +diff --git a/arch/x86/kernel/asm-offsets.c b/arch/x86/kernel/asm-offsets.c +index cd360a5e0dca..676b7cf4b62b 100644 +--- a/arch/x86/kernel/asm-offsets.c ++++ b/arch/x86/kernel/asm-offsets.c +@@ -97,6 +97,6 @@ void common(void) { + /* Layout info for cpu_entry_area */ + OFFSET(CPU_ENTRY_AREA_tss, cpu_entry_area, tss); + OFFSET(CPU_ENTRY_AREA_entry_trampoline, cpu_entry_area, entry_trampoline); +- OFFSET(CPU_ENTRY_AREA_SYSENTER_stack, cpu_entry_area, SYSENTER_stack_page); +- DEFINE(SIZEOF_SYSENTER_stack, sizeof(struct SYSENTER_stack)); ++ OFFSET(CPU_ENTRY_AREA_entry_stack, cpu_entry_area, entry_stack_page); ++ DEFINE(SIZEOF_entry_stack, sizeof(struct entry_stack)); + } +diff --git a/arch/x86/kernel/asm-offsets_32.c b/arch/x86/kernel/asm-offsets_32.c +index 7d20d9c0b3d6..fa1261eefa16 100644 +--- a/arch/x86/kernel/asm-offsets_32.c ++++ b/arch/x86/kernel/asm-offsets_32.c +@@ -48,7 +48,7 @@ void foo(void) + + /* Offset from the sysenter stack to tss.sp0 */ + DEFINE(TSS_sysenter_sp0, offsetof(struct cpu_entry_area, tss.x86_tss.sp0) - +- offsetofend(struct cpu_entry_area, SYSENTER_stack_page.stack)); ++ offsetofend(struct cpu_entry_area, entry_stack_page.stack)); + + #ifdef CONFIG_CC_STACKPROTECTOR + BLANK(); +diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c +index 034900623adf..8ddcfa4d4165 100644 +--- a/arch/x86/kernel/cpu/common.c ++++ b/arch/x86/kernel/cpu/common.c +@@ -482,102 +482,8 @@ static const unsigned int exception_stack_sizes[N_EXCEPTION_STACKS] = { + [0 ... N_EXCEPTION_STACKS - 1] = EXCEPTION_STKSZ, + [DEBUG_STACK - 1] = DEBUG_STKSZ + }; +- +-static DEFINE_PER_CPU_PAGE_ALIGNED(char, exception_stacks +- [(N_EXCEPTION_STACKS - 1) * EXCEPTION_STKSZ + DEBUG_STKSZ]); +-#endif +- +-static DEFINE_PER_CPU_PAGE_ALIGNED(struct SYSENTER_stack_page, +- SYSENTER_stack_storage); +- +-static void __init +-set_percpu_fixmap_pages(int idx, void *ptr, int pages, pgprot_t prot) +-{ +- for ( ; pages; pages--, idx--, ptr += PAGE_SIZE) +- __set_fixmap(idx, per_cpu_ptr_to_phys(ptr), prot); +-} +- +-/* Setup the fixmap mappings only once per-processor */ +-static void __init setup_cpu_entry_area(int cpu) +-{ +-#ifdef CONFIG_X86_64 +- extern char _entry_trampoline[]; +- +- /* On 64-bit systems, we use a read-only fixmap GDT and TSS. */ +- pgprot_t gdt_prot = PAGE_KERNEL_RO; +- pgprot_t tss_prot = PAGE_KERNEL_RO; +-#else +- /* +- * On native 32-bit systems, the GDT cannot be read-only because +- * our double fault handler uses a task gate, and entering through +- * a task gate needs to change an available TSS to busy. If the +- * GDT is read-only, that will triple fault. The TSS cannot be +- * read-only because the CPU writes to it on task switches. +- * +- * On Xen PV, the GDT must be read-only because the hypervisor +- * requires it. +- */ +- pgprot_t gdt_prot = boot_cpu_has(X86_FEATURE_XENPV) ? +- PAGE_KERNEL_RO : PAGE_KERNEL; +- pgprot_t tss_prot = PAGE_KERNEL; +-#endif +- +- __set_fixmap(get_cpu_entry_area_index(cpu, gdt), get_cpu_gdt_paddr(cpu), gdt_prot); +- set_percpu_fixmap_pages(get_cpu_entry_area_index(cpu, SYSENTER_stack_page), +- per_cpu_ptr(&SYSENTER_stack_storage, cpu), 1, +- PAGE_KERNEL); +- +- /* +- * The Intel SDM says (Volume 3, 7.2.1): +- * +- * Avoid placing a page boundary in the part of the TSS that the +- * processor reads during a task switch (the first 104 bytes). The +- * processor may not correctly perform address translations if a +- * boundary occurs in this area. During a task switch, the processor +- * reads and writes into the first 104 bytes of each TSS (using +- * contiguous physical addresses beginning with the physical address +- * of the first byte of the TSS). So, after TSS access begins, if +- * part of the 104 bytes is not physically contiguous, the processor +- * will access incorrect information without generating a page-fault +- * exception. +- * +- * There are also a lot of errata involving the TSS spanning a page +- * boundary. Assert that we're not doing that. +- */ +- BUILD_BUG_ON((offsetof(struct tss_struct, x86_tss) ^ +- offsetofend(struct tss_struct, x86_tss)) & PAGE_MASK); +- BUILD_BUG_ON(sizeof(struct tss_struct) % PAGE_SIZE != 0); +- set_percpu_fixmap_pages(get_cpu_entry_area_index(cpu, tss), +- &per_cpu(cpu_tss_rw, cpu), +- sizeof(struct tss_struct) / PAGE_SIZE, +- tss_prot); +- +-#ifdef CONFIG_X86_32 +- per_cpu(cpu_entry_area, cpu) = get_cpu_entry_area(cpu); + #endif + +-#ifdef CONFIG_X86_64 +- BUILD_BUG_ON(sizeof(exception_stacks) % PAGE_SIZE != 0); +- BUILD_BUG_ON(sizeof(exception_stacks) != +- sizeof(((struct cpu_entry_area *)0)->exception_stacks)); +- set_percpu_fixmap_pages(get_cpu_entry_area_index(cpu, exception_stacks), +- &per_cpu(exception_stacks, cpu), +- sizeof(exception_stacks) / PAGE_SIZE, +- PAGE_KERNEL); +- +- __set_fixmap(get_cpu_entry_area_index(cpu, entry_trampoline), +- __pa_symbol(_entry_trampoline), PAGE_KERNEL_RX); +-#endif +-} +- +-void __init setup_cpu_entry_areas(void) +-{ +- unsigned int cpu; +- +- for_each_possible_cpu(cpu) +- setup_cpu_entry_area(cpu); +-} +- + /* Load the original GDT from the per-cpu structure */ + void load_direct_gdt(int cpu) + { +@@ -1323,7 +1229,7 @@ void enable_sep_cpu(void) + + tss->x86_tss.ss1 = __KERNEL_CS; + wrmsr(MSR_IA32_SYSENTER_CS, tss->x86_tss.ss1, 0); +- wrmsr(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_SYSENTER_stack(cpu) + 1), 0); ++ wrmsr(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_entry_stack(cpu) + 1), 0); + wrmsr(MSR_IA32_SYSENTER_EIP, (unsigned long)entry_SYSENTER_32, 0); + + put_cpu(); +@@ -1440,7 +1346,7 @@ void syscall_init(void) + * AMD doesn't allow SYSENTER in long mode (either 32- or 64-bit). + */ + wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)__KERNEL_CS); +- wrmsrl_safe(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_SYSENTER_stack(cpu) + 1)); ++ wrmsrl_safe(MSR_IA32_SYSENTER_ESP, (unsigned long)(cpu_entry_stack(cpu) + 1)); + wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)entry_SYSENTER_compat); + #else + wrmsrl(MSR_CSTAR, (unsigned long)ignore_sysret); +@@ -1655,7 +1561,7 @@ void cpu_init(void) + */ + set_tss_desc(cpu, &get_cpu_entry_area(cpu)->tss.x86_tss); + load_TR_desc(); +- load_sp0((unsigned long)(cpu_SYSENTER_stack(cpu) + 1)); ++ load_sp0((unsigned long)(cpu_entry_stack(cpu) + 1)); + + load_mm_ldt(&init_mm); + +diff --git a/arch/x86/kernel/cpu/microcode/intel.c b/arch/x86/kernel/cpu/microcode/intel.c +index 7dbcb7adf797..8ccdca6d3f9e 100644 +--- a/arch/x86/kernel/cpu/microcode/intel.c ++++ b/arch/x86/kernel/cpu/microcode/intel.c +@@ -565,15 +565,6 @@ static void print_ucode(struct ucode_cpu_info *uci) + } + #else + +-/* +- * Flush global tlb. We only do this in x86_64 where paging has been enabled +- * already and PGE should be enabled as well. +- */ +-static inline void flush_tlb_early(void) +-{ +- __native_flush_tlb_global_irq_disabled(); +-} +- + static inline void print_ucode(struct ucode_cpu_info *uci) + { + struct microcode_intel *mc; +@@ -602,10 +593,6 @@ static int apply_microcode_early(struct ucode_cpu_info *uci, bool early) + if (rev != mc->hdr.rev) + return -1; + +-#ifdef CONFIG_X86_64 +- /* Flush global tlb. This is precaution. */ +- flush_tlb_early(); +-#endif + uci->cpu_sig.rev = rev; + + if (early) +diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c +index bbd6d986e2d0..36b17e0febe8 100644 +--- a/arch/x86/kernel/dumpstack.c ++++ b/arch/x86/kernel/dumpstack.c +@@ -18,6 +18,7 @@ + #include + #include + ++#include + #include + #include + +@@ -43,9 +44,9 @@ bool in_task_stack(unsigned long *stack, struct task_struct *task, + return true; + } + +-bool in_sysenter_stack(unsigned long *stack, struct stack_info *info) ++bool in_entry_stack(unsigned long *stack, struct stack_info *info) + { +- struct SYSENTER_stack *ss = cpu_SYSENTER_stack(smp_processor_id()); ++ struct entry_stack *ss = cpu_entry_stack(smp_processor_id()); + + void *begin = ss; + void *end = ss + 1; +@@ -53,7 +54,7 @@ bool in_sysenter_stack(unsigned long *stack, struct stack_info *info) + if ((void *)stack < begin || (void *)stack >= end) + return false; + +- info->type = STACK_TYPE_SYSENTER; ++ info->type = STACK_TYPE_ENTRY; + info->begin = begin; + info->end = end; + info->next_sp = NULL; +@@ -111,13 +112,13 @@ void show_trace_log_lvl(struct task_struct *task, struct pt_regs *regs, + * - task stack + * - interrupt stack + * - HW exception stacks (double fault, nmi, debug, mce) +- * - SYSENTER stack ++ * - entry stack + * + * x86-32 can have up to four stacks: + * - task stack + * - softirq stack + * - hardirq stack +- * - SYSENTER stack ++ * - entry stack + */ + for (regs = NULL; stack; stack = PTR_ALIGN(stack_info.next_sp, sizeof(long))) { + const char *stack_name; +diff --git a/arch/x86/kernel/dumpstack_32.c b/arch/x86/kernel/dumpstack_32.c +index 5ff13a6b3680..04170f63e3a1 100644 +--- a/arch/x86/kernel/dumpstack_32.c ++++ b/arch/x86/kernel/dumpstack_32.c +@@ -26,8 +26,8 @@ const char *stack_type_name(enum stack_type type) + if (type == STACK_TYPE_SOFTIRQ) + return "SOFTIRQ"; + +- if (type == STACK_TYPE_SYSENTER) +- return "SYSENTER"; ++ if (type == STACK_TYPE_ENTRY) ++ return "ENTRY_TRAMPOLINE"; + + return NULL; + } +@@ -96,7 +96,7 @@ int get_stack_info(unsigned long *stack, struct task_struct *task, + if (task != current) + goto unknown; + +- if (in_sysenter_stack(stack, info)) ++ if (in_entry_stack(stack, info)) + goto recursion_check; + + if (in_hardirq_stack(stack, info)) +diff --git a/arch/x86/kernel/dumpstack_64.c b/arch/x86/kernel/dumpstack_64.c +index abc828f8c297..563e28d14f2c 100644 +--- a/arch/x86/kernel/dumpstack_64.c ++++ b/arch/x86/kernel/dumpstack_64.c +@@ -37,8 +37,14 @@ const char *stack_type_name(enum stack_type type) + if (type == STACK_TYPE_IRQ) + return "IRQ"; + +- if (type == STACK_TYPE_SYSENTER) +- return "SYSENTER"; ++ if (type == STACK_TYPE_ENTRY) { ++ /* ++ * On 64-bit, we have a generic entry stack that we ++ * use for all the kernel entry points, including ++ * SYSENTER. ++ */ ++ return "ENTRY_TRAMPOLINE"; ++ } + + if (type >= STACK_TYPE_EXCEPTION && type <= STACK_TYPE_EXCEPTION_LAST) + return exception_stack_names[type - STACK_TYPE_EXCEPTION]; +@@ -118,7 +124,7 @@ int get_stack_info(unsigned long *stack, struct task_struct *task, + if (in_irq_stack(stack, info)) + goto recursion_check; + +- if (in_sysenter_stack(stack, info)) ++ if (in_entry_stack(stack, info)) + goto recursion_check; + + goto unknown; +diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c +index 1c1eae961340..a6b5d62f45a7 100644 +--- a/arch/x86/kernel/ldt.c ++++ b/arch/x86/kernel/ldt.c +@@ -5,6 +5,11 @@ + * Copyright (C) 2002 Andi Kleen + * + * This handles calls from both 32bit and 64bit mode. ++ * ++ * Lock order: ++ * contex.ldt_usr_sem ++ * mmap_sem ++ * context.lock + */ + + #include +@@ -42,7 +47,7 @@ static void refresh_ldt_segments(void) + #endif + } + +-/* context.lock is held for us, so we don't need any locking. */ ++/* context.lock is held by the task which issued the smp function call */ + static void flush_ldt(void *__mm) + { + struct mm_struct *mm = __mm; +@@ -99,15 +104,17 @@ static void finalize_ldt_struct(struct ldt_struct *ldt) + paravirt_alloc_ldt(ldt->entries, ldt->nr_entries); + } + +-/* context.lock is held */ +-static void install_ldt(struct mm_struct *current_mm, +- struct ldt_struct *ldt) ++static void install_ldt(struct mm_struct *mm, struct ldt_struct *ldt) + { ++ mutex_lock(&mm->context.lock); ++ + /* Synchronizes with READ_ONCE in load_mm_ldt. */ +- smp_store_release(¤t_mm->context.ldt, ldt); ++ smp_store_release(&mm->context.ldt, ldt); + +- /* Activate the LDT for all CPUs using current_mm. */ +- on_each_cpu_mask(mm_cpumask(current_mm), flush_ldt, current_mm, true); ++ /* Activate the LDT for all CPUs using currents mm. */ ++ on_each_cpu_mask(mm_cpumask(mm), flush_ldt, mm, true); ++ ++ mutex_unlock(&mm->context.lock); + } + + static void free_ldt_struct(struct ldt_struct *ldt) +@@ -124,27 +131,20 @@ static void free_ldt_struct(struct ldt_struct *ldt) + } + + /* +- * we do not have to muck with descriptors here, that is +- * done in switch_mm() as needed. ++ * Called on fork from arch_dup_mmap(). Just copy the current LDT state, ++ * the new task is not running, so nothing can be installed. + */ +-int init_new_context_ldt(struct task_struct *tsk, struct mm_struct *mm) ++int ldt_dup_context(struct mm_struct *old_mm, struct mm_struct *mm) + { + struct ldt_struct *new_ldt; +- struct mm_struct *old_mm; + int retval = 0; + +- mutex_init(&mm->context.lock); +- old_mm = current->mm; +- if (!old_mm) { +- mm->context.ldt = NULL; ++ if (!old_mm) + return 0; +- } + + mutex_lock(&old_mm->context.lock); +- if (!old_mm->context.ldt) { +- mm->context.ldt = NULL; ++ if (!old_mm->context.ldt) + goto out_unlock; +- } + + new_ldt = alloc_ldt_struct(old_mm->context.ldt->nr_entries); + if (!new_ldt) { +@@ -180,7 +180,7 @@ static int read_ldt(void __user *ptr, unsigned long bytecount) + unsigned long entries_size; + int retval; + +- mutex_lock(&mm->context.lock); ++ down_read(&mm->context.ldt_usr_sem); + + if (!mm->context.ldt) { + retval = 0; +@@ -209,7 +209,7 @@ static int read_ldt(void __user *ptr, unsigned long bytecount) + retval = bytecount; + + out_unlock: +- mutex_unlock(&mm->context.lock); ++ up_read(&mm->context.ldt_usr_sem); + return retval; + } + +@@ -269,7 +269,8 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode) + ldt.avl = 0; + } + +- mutex_lock(&mm->context.lock); ++ if (down_write_killable(&mm->context.ldt_usr_sem)) ++ return -EINTR; + + old_ldt = mm->context.ldt; + old_nr_entries = old_ldt ? old_ldt->nr_entries : 0; +@@ -291,7 +292,7 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode) + error = 0; + + out_unlock: +- mutex_unlock(&mm->context.lock); ++ up_write(&mm->context.ldt_usr_sem); + out: + return error; + } +diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c +index 142126ab5aae..12bf07d44dfe 100644 +--- a/arch/x86/kernel/smpboot.c ++++ b/arch/x86/kernel/smpboot.c +@@ -990,12 +990,8 @@ static int do_boot_cpu(int apicid, int cpu, struct task_struct *idle, + initial_code = (unsigned long)start_secondary; + initial_stack = idle->thread.sp; + +- /* +- * Enable the espfix hack for this CPU +- */ +-#ifdef CONFIG_X86_ESPFIX64 ++ /* Enable the espfix hack for this CPU */ + init_espfix_ap(cpu); +-#endif + + /* So we see what's up */ + announce_cpu(cpu, apicid); +diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c +index 74136fd16f49..7c16fe0b60c2 100644 +--- a/arch/x86/kernel/traps.c ++++ b/arch/x86/kernel/traps.c +@@ -52,6 +52,7 @@ + #include + #include + #include ++#include + #include + #include + #include +@@ -950,8 +951,9 @@ void __init trap_init(void) + * "sidt" instruction will not leak the location of the kernel, and + * to defend the IDT against arbitrary memory write vulnerabilities. + * It will be reloaded in cpu_init() */ +- __set_fixmap(FIX_RO_IDT, __pa_symbol(idt_table), PAGE_KERNEL_RO); +- idt_descr.address = fix_to_virt(FIX_RO_IDT); ++ cea_set_pte(CPU_ENTRY_AREA_RO_IDT_VADDR, __pa_symbol(idt_table), ++ PAGE_KERNEL_RO); ++ idt_descr.address = CPU_ENTRY_AREA_RO_IDT; + + /* + * Should be a barrier for any external CPU state: +diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c +index d90cdc77e077..7bbb5da2b49d 100644 +--- a/arch/x86/kvm/emulate.c ++++ b/arch/x86/kvm/emulate.c +@@ -2404,9 +2404,21 @@ static int rsm_load_seg_64(struct x86_emulate_ctxt *ctxt, u64 smbase, int n) + } + + static int rsm_enter_protected_mode(struct x86_emulate_ctxt *ctxt, +- u64 cr0, u64 cr4) ++ u64 cr0, u64 cr3, u64 cr4) + { + int bad; ++ u64 pcid; ++ ++ /* In order to later set CR4.PCIDE, CR3[11:0] must be zero. */ ++ pcid = 0; ++ if (cr4 & X86_CR4_PCIDE) { ++ pcid = cr3 & 0xfff; ++ cr3 &= ~0xfff; ++ } ++ ++ bad = ctxt->ops->set_cr(ctxt, 3, cr3); ++ if (bad) ++ return X86EMUL_UNHANDLEABLE; + + /* + * First enable PAE, long mode needs it before CR0.PG = 1 is set. +@@ -2425,6 +2437,12 @@ static int rsm_enter_protected_mode(struct x86_emulate_ctxt *ctxt, + bad = ctxt->ops->set_cr(ctxt, 4, cr4); + if (bad) + return X86EMUL_UNHANDLEABLE; ++ if (pcid) { ++ bad = ctxt->ops->set_cr(ctxt, 3, cr3 | pcid); ++ if (bad) ++ return X86EMUL_UNHANDLEABLE; ++ } ++ + } + + return X86EMUL_CONTINUE; +@@ -2435,11 +2453,11 @@ static int rsm_load_state_32(struct x86_emulate_ctxt *ctxt, u64 smbase) + struct desc_struct desc; + struct desc_ptr dt; + u16 selector; +- u32 val, cr0, cr4; ++ u32 val, cr0, cr3, cr4; + int i; + + cr0 = GET_SMSTATE(u32, smbase, 0x7ffc); +- ctxt->ops->set_cr(ctxt, 3, GET_SMSTATE(u32, smbase, 0x7ff8)); ++ cr3 = GET_SMSTATE(u32, smbase, 0x7ff8); + ctxt->eflags = GET_SMSTATE(u32, smbase, 0x7ff4) | X86_EFLAGS_FIXED; + ctxt->_eip = GET_SMSTATE(u32, smbase, 0x7ff0); + +@@ -2481,14 +2499,14 @@ static int rsm_load_state_32(struct x86_emulate_ctxt *ctxt, u64 smbase) + + ctxt->ops->set_smbase(ctxt, GET_SMSTATE(u32, smbase, 0x7ef8)); + +- return rsm_enter_protected_mode(ctxt, cr0, cr4); ++ return rsm_enter_protected_mode(ctxt, cr0, cr3, cr4); + } + + static int rsm_load_state_64(struct x86_emulate_ctxt *ctxt, u64 smbase) + { + struct desc_struct desc; + struct desc_ptr dt; +- u64 val, cr0, cr4; ++ u64 val, cr0, cr3, cr4; + u32 base3; + u16 selector; + int i, r; +@@ -2505,7 +2523,7 @@ static int rsm_load_state_64(struct x86_emulate_ctxt *ctxt, u64 smbase) + ctxt->ops->set_dr(ctxt, 7, (val & DR7_VOLATILE) | DR7_FIXED_1); + + cr0 = GET_SMSTATE(u64, smbase, 0x7f58); +- ctxt->ops->set_cr(ctxt, 3, GET_SMSTATE(u64, smbase, 0x7f50)); ++ cr3 = GET_SMSTATE(u64, smbase, 0x7f50); + cr4 = GET_SMSTATE(u64, smbase, 0x7f48); + ctxt->ops->set_smbase(ctxt, GET_SMSTATE(u32, smbase, 0x7f00)); + val = GET_SMSTATE(u64, smbase, 0x7ed0); +@@ -2533,7 +2551,7 @@ static int rsm_load_state_64(struct x86_emulate_ctxt *ctxt, u64 smbase) + dt.address = GET_SMSTATE(u64, smbase, 0x7e68); + ctxt->ops->set_gdt(ctxt, &dt); + +- r = rsm_enter_protected_mode(ctxt, cr0, cr4); ++ r = rsm_enter_protected_mode(ctxt, cr0, cr3, cr4); + if (r != X86EMUL_CONTINUE) + return r; + +diff --git a/arch/x86/kvm/mmu.c b/arch/x86/kvm/mmu.c +index 13ebeedcec07..0fce8d73403c 100644 +--- a/arch/x86/kvm/mmu.c ++++ b/arch/x86/kvm/mmu.c +@@ -3382,7 +3382,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) + spin_lock(&vcpu->kvm->mmu_lock); + if(make_mmu_pages_available(vcpu) < 0) { + spin_unlock(&vcpu->kvm->mmu_lock); +- return 1; ++ return -ENOSPC; + } + sp = kvm_mmu_get_page(vcpu, 0, 0, + vcpu->arch.mmu.shadow_root_level, 1, ACC_ALL); +@@ -3397,7 +3397,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu) + spin_lock(&vcpu->kvm->mmu_lock); + if (make_mmu_pages_available(vcpu) < 0) { + spin_unlock(&vcpu->kvm->mmu_lock); +- return 1; ++ return -ENOSPC; + } + sp = kvm_mmu_get_page(vcpu, i << (30 - PAGE_SHIFT), + i << 30, PT32_ROOT_LEVEL, 1, ACC_ALL); +@@ -3437,7 +3437,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) + spin_lock(&vcpu->kvm->mmu_lock); + if (make_mmu_pages_available(vcpu) < 0) { + spin_unlock(&vcpu->kvm->mmu_lock); +- return 1; ++ return -ENOSPC; + } + sp = kvm_mmu_get_page(vcpu, root_gfn, 0, + vcpu->arch.mmu.shadow_root_level, 0, ACC_ALL); +@@ -3474,7 +3474,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) + spin_lock(&vcpu->kvm->mmu_lock); + if (make_mmu_pages_available(vcpu) < 0) { + spin_unlock(&vcpu->kvm->mmu_lock); +- return 1; ++ return -ENOSPC; + } + sp = kvm_mmu_get_page(vcpu, root_gfn, i << 30, PT32_ROOT_LEVEL, + 0, ACC_ALL); +diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c +index df62cdc7a258..075619a92ce7 100644 +--- a/arch/x86/kvm/x86.c ++++ b/arch/x86/kvm/x86.c +@@ -7359,7 +7359,7 @@ int kvm_arch_vcpu_ioctl_set_regs(struct kvm_vcpu *vcpu, struct kvm_regs *regs) + #endif + + kvm_rip_write(vcpu, regs->rip); +- kvm_set_rflags(vcpu, regs->rflags); ++ kvm_set_rflags(vcpu, regs->rflags | X86_EFLAGS_FIXED); + + vcpu->arch.exception.pending = false; + +diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.txt +index c4d55919fac1..e0b85930dd77 100644 +--- a/arch/x86/lib/x86-opcode-map.txt ++++ b/arch/x86/lib/x86-opcode-map.txt +@@ -607,7 +607,7 @@ fb: psubq Pq,Qq | vpsubq Vx,Hx,Wx (66),(v1) + fc: paddb Pq,Qq | vpaddb Vx,Hx,Wx (66),(v1) + fd: paddw Pq,Qq | vpaddw Vx,Hx,Wx (66),(v1) + fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1) +-ff: ++ff: UD0 + EndTable + + Table: 3-byte opcode 1 (0x0f 0x38) +@@ -717,7 +717,7 @@ AVXcode: 2 + 7e: vpermt2d/q Vx,Hx,Wx (66),(ev) + 7f: vpermt2ps/d Vx,Hx,Wx (66),(ev) + 80: INVEPT Gy,Mdq (66) +-81: INVPID Gy,Mdq (66) ++81: INVVPID Gy,Mdq (66) + 82: INVPCID Gy,Mdq (66) + 83: vpmultishiftqb Vx,Hx,Wx (66),(ev) + 88: vexpandps/d Vpd,Wpd (66),(ev) +@@ -970,6 +970,15 @@ GrpTable: Grp9 + EndTable + + GrpTable: Grp10 ++# all are UD1 ++0: UD1 ++1: UD1 ++2: UD1 ++3: UD1 ++4: UD1 ++5: UD1 ++6: UD1 ++7: UD1 + EndTable + + # Grp11A and Grp11B are expressed as Grp11 in Intel SDM +diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile +index 7ba7f3d7f477..2e0017af8f9b 100644 +--- a/arch/x86/mm/Makefile ++++ b/arch/x86/mm/Makefile +@@ -10,7 +10,7 @@ CFLAGS_REMOVE_mem_encrypt.o = -pg + endif + + obj-y := init.o init_$(BITS).o fault.o ioremap.o extable.o pageattr.o mmap.o \ +- pat.o pgtable.o physaddr.o setup_nx.o tlb.o ++ pat.o pgtable.o physaddr.o setup_nx.o tlb.o cpu_entry_area.o + + # Make sure __phys_addr has no stackprotector + nostackp := $(call cc-option, -fno-stack-protector) +diff --git a/arch/x86/mm/cpu_entry_area.c b/arch/x86/mm/cpu_entry_area.c +new file mode 100644 +index 000000000000..fe814fd5e014 +--- /dev/null ++++ b/arch/x86/mm/cpu_entry_area.c +@@ -0,0 +1,139 @@ ++// SPDX-License-Identifier: GPL-2.0 ++ ++#include ++#include ++ ++#include ++#include ++#include ++#include ++ ++static DEFINE_PER_CPU_PAGE_ALIGNED(struct entry_stack_page, entry_stack_storage); ++ ++#ifdef CONFIG_X86_64 ++static DEFINE_PER_CPU_PAGE_ALIGNED(char, exception_stacks ++ [(N_EXCEPTION_STACKS - 1) * EXCEPTION_STKSZ + DEBUG_STKSZ]); ++#endif ++ ++struct cpu_entry_area *get_cpu_entry_area(int cpu) ++{ ++ unsigned long va = CPU_ENTRY_AREA_PER_CPU + cpu * CPU_ENTRY_AREA_SIZE; ++ BUILD_BUG_ON(sizeof(struct cpu_entry_area) % PAGE_SIZE != 0); ++ ++ return (struct cpu_entry_area *) va; ++} ++EXPORT_SYMBOL(get_cpu_entry_area); ++ ++void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags) ++{ ++ unsigned long va = (unsigned long) cea_vaddr; ++ ++ set_pte_vaddr(va, pfn_pte(pa >> PAGE_SHIFT, flags)); ++} ++ ++static void __init ++cea_map_percpu_pages(void *cea_vaddr, void *ptr, int pages, pgprot_t prot) ++{ ++ for ( ; pages; pages--, cea_vaddr+= PAGE_SIZE, ptr += PAGE_SIZE) ++ cea_set_pte(cea_vaddr, per_cpu_ptr_to_phys(ptr), prot); ++} ++ ++/* Setup the fixmap mappings only once per-processor */ ++static void __init setup_cpu_entry_area(int cpu) ++{ ++#ifdef CONFIG_X86_64 ++ extern char _entry_trampoline[]; ++ ++ /* On 64-bit systems, we use a read-only fixmap GDT and TSS. */ ++ pgprot_t gdt_prot = PAGE_KERNEL_RO; ++ pgprot_t tss_prot = PAGE_KERNEL_RO; ++#else ++ /* ++ * On native 32-bit systems, the GDT cannot be read-only because ++ * our double fault handler uses a task gate, and entering through ++ * a task gate needs to change an available TSS to busy. If the ++ * GDT is read-only, that will triple fault. The TSS cannot be ++ * read-only because the CPU writes to it on task switches. ++ * ++ * On Xen PV, the GDT must be read-only because the hypervisor ++ * requires it. ++ */ ++ pgprot_t gdt_prot = boot_cpu_has(X86_FEATURE_XENPV) ? ++ PAGE_KERNEL_RO : PAGE_KERNEL; ++ pgprot_t tss_prot = PAGE_KERNEL; ++#endif ++ ++ cea_set_pte(&get_cpu_entry_area(cpu)->gdt, get_cpu_gdt_paddr(cpu), ++ gdt_prot); ++ ++ cea_map_percpu_pages(&get_cpu_entry_area(cpu)->entry_stack_page, ++ per_cpu_ptr(&entry_stack_storage, cpu), 1, ++ PAGE_KERNEL); ++ ++ /* ++ * The Intel SDM says (Volume 3, 7.2.1): ++ * ++ * Avoid placing a page boundary in the part of the TSS that the ++ * processor reads during a task switch (the first 104 bytes). The ++ * processor may not correctly perform address translations if a ++ * boundary occurs in this area. During a task switch, the processor ++ * reads and writes into the first 104 bytes of each TSS (using ++ * contiguous physical addresses beginning with the physical address ++ * of the first byte of the TSS). So, after TSS access begins, if ++ * part of the 104 bytes is not physically contiguous, the processor ++ * will access incorrect information without generating a page-fault ++ * exception. ++ * ++ * There are also a lot of errata involving the TSS spanning a page ++ * boundary. Assert that we're not doing that. ++ */ ++ BUILD_BUG_ON((offsetof(struct tss_struct, x86_tss) ^ ++ offsetofend(struct tss_struct, x86_tss)) & PAGE_MASK); ++ BUILD_BUG_ON(sizeof(struct tss_struct) % PAGE_SIZE != 0); ++ cea_map_percpu_pages(&get_cpu_entry_area(cpu)->tss, ++ &per_cpu(cpu_tss_rw, cpu), ++ sizeof(struct tss_struct) / PAGE_SIZE, tss_prot); ++ ++#ifdef CONFIG_X86_32 ++ per_cpu(cpu_entry_area, cpu) = get_cpu_entry_area(cpu); ++#endif ++ ++#ifdef CONFIG_X86_64 ++ BUILD_BUG_ON(sizeof(exception_stacks) % PAGE_SIZE != 0); ++ BUILD_BUG_ON(sizeof(exception_stacks) != ++ sizeof(((struct cpu_entry_area *)0)->exception_stacks)); ++ cea_map_percpu_pages(&get_cpu_entry_area(cpu)->exception_stacks, ++ &per_cpu(exception_stacks, cpu), ++ sizeof(exception_stacks) / PAGE_SIZE, PAGE_KERNEL); ++ ++ cea_set_pte(&get_cpu_entry_area(cpu)->entry_trampoline, ++ __pa_symbol(_entry_trampoline), PAGE_KERNEL_RX); ++#endif ++} ++ ++static __init void setup_cpu_entry_area_ptes(void) ++{ ++#ifdef CONFIG_X86_32 ++ unsigned long start, end; ++ ++ BUILD_BUG_ON(CPU_ENTRY_AREA_PAGES * PAGE_SIZE < CPU_ENTRY_AREA_MAP_SIZE); ++ BUG_ON(CPU_ENTRY_AREA_BASE & ~PMD_MASK); ++ ++ start = CPU_ENTRY_AREA_BASE; ++ end = start + CPU_ENTRY_AREA_MAP_SIZE; ++ ++ /* Careful here: start + PMD_SIZE might wrap around */ ++ for (; start < end && start >= CPU_ENTRY_AREA_BASE; start += PMD_SIZE) ++ populate_extra_pte(start); ++#endif ++} ++ ++void __init setup_cpu_entry_areas(void) ++{ ++ unsigned int cpu; ++ ++ setup_cpu_entry_area_ptes(); ++ ++ for_each_possible_cpu(cpu) ++ setup_cpu_entry_area(cpu); ++} +diff --git a/arch/x86/mm/dump_pagetables.c b/arch/x86/mm/dump_pagetables.c +index 5e3ac6fe6c9e..43dedbfb7257 100644 +--- a/arch/x86/mm/dump_pagetables.c ++++ b/arch/x86/mm/dump_pagetables.c +@@ -44,10 +44,12 @@ struct addr_marker { + unsigned long max_lines; + }; + +-/* indices for address_markers; keep sync'd w/ address_markers below */ ++/* Address space markers hints */ ++ ++#ifdef CONFIG_X86_64 ++ + enum address_markers_idx { + USER_SPACE_NR = 0, +-#ifdef CONFIG_X86_64 + KERNEL_SPACE_NR, + LOW_KERNEL_NR, + VMALLOC_START_NR, +@@ -56,56 +58,74 @@ enum address_markers_idx { + KASAN_SHADOW_START_NR, + KASAN_SHADOW_END_NR, + #endif +-# ifdef CONFIG_X86_ESPFIX64 ++ CPU_ENTRY_AREA_NR, ++#ifdef CONFIG_X86_ESPFIX64 + ESPFIX_START_NR, +-# endif ++#endif ++#ifdef CONFIG_EFI ++ EFI_END_NR, ++#endif + HIGH_KERNEL_NR, + MODULES_VADDR_NR, + MODULES_END_NR, +-#else ++ FIXADDR_START_NR, ++ END_OF_SPACE_NR, ++}; ++ ++static struct addr_marker address_markers[] = { ++ [USER_SPACE_NR] = { 0, "User Space" }, ++ [KERNEL_SPACE_NR] = { (1UL << 63), "Kernel Space" }, ++ [LOW_KERNEL_NR] = { 0UL, "Low Kernel Mapping" }, ++ [VMALLOC_START_NR] = { 0UL, "vmalloc() Area" }, ++ [VMEMMAP_START_NR] = { 0UL, "Vmemmap" }, ++#ifdef CONFIG_KASAN ++ [KASAN_SHADOW_START_NR] = { KASAN_SHADOW_START, "KASAN shadow" }, ++ [KASAN_SHADOW_END_NR] = { KASAN_SHADOW_END, "KASAN shadow end" }, ++#endif ++ [CPU_ENTRY_AREA_NR] = { CPU_ENTRY_AREA_BASE,"CPU entry Area" }, ++#ifdef CONFIG_X86_ESPFIX64 ++ [ESPFIX_START_NR] = { ESPFIX_BASE_ADDR, "ESPfix Area", 16 }, ++#endif ++#ifdef CONFIG_EFI ++ [EFI_END_NR] = { EFI_VA_END, "EFI Runtime Services" }, ++#endif ++ [HIGH_KERNEL_NR] = { __START_KERNEL_map, "High Kernel Mapping" }, ++ [MODULES_VADDR_NR] = { MODULES_VADDR, "Modules" }, ++ [MODULES_END_NR] = { MODULES_END, "End Modules" }, ++ [FIXADDR_START_NR] = { FIXADDR_START, "Fixmap Area" }, ++ [END_OF_SPACE_NR] = { -1, NULL } ++}; ++ ++#else /* CONFIG_X86_64 */ ++ ++enum address_markers_idx { ++ USER_SPACE_NR = 0, + KERNEL_SPACE_NR, + VMALLOC_START_NR, + VMALLOC_END_NR, +-# ifdef CONFIG_HIGHMEM ++#ifdef CONFIG_HIGHMEM + PKMAP_BASE_NR, +-# endif +- FIXADDR_START_NR, + #endif ++ CPU_ENTRY_AREA_NR, ++ FIXADDR_START_NR, ++ END_OF_SPACE_NR, + }; + +-/* Address space markers hints */ + static struct addr_marker address_markers[] = { +- { 0, "User Space" }, +-#ifdef CONFIG_X86_64 +- { 0x8000000000000000UL, "Kernel Space" }, +- { 0/* PAGE_OFFSET */, "Low Kernel Mapping" }, +- { 0/* VMALLOC_START */, "vmalloc() Area" }, +- { 0/* VMEMMAP_START */, "Vmemmap" }, +-#ifdef CONFIG_KASAN +- { KASAN_SHADOW_START, "KASAN shadow" }, +- { KASAN_SHADOW_END, "KASAN shadow end" }, ++ [USER_SPACE_NR] = { 0, "User Space" }, ++ [KERNEL_SPACE_NR] = { PAGE_OFFSET, "Kernel Mapping" }, ++ [VMALLOC_START_NR] = { 0UL, "vmalloc() Area" }, ++ [VMALLOC_END_NR] = { 0UL, "vmalloc() End" }, ++#ifdef CONFIG_HIGHMEM ++ [PKMAP_BASE_NR] = { 0UL, "Persistent kmap() Area" }, + #endif +-# ifdef CONFIG_X86_ESPFIX64 +- { ESPFIX_BASE_ADDR, "ESPfix Area", 16 }, +-# endif +-# ifdef CONFIG_EFI +- { EFI_VA_END, "EFI Runtime Services" }, +-# endif +- { __START_KERNEL_map, "High Kernel Mapping" }, +- { MODULES_VADDR, "Modules" }, +- { MODULES_END, "End Modules" }, +-#else +- { PAGE_OFFSET, "Kernel Mapping" }, +- { 0/* VMALLOC_START */, "vmalloc() Area" }, +- { 0/*VMALLOC_END*/, "vmalloc() End" }, +-# ifdef CONFIG_HIGHMEM +- { 0/*PKMAP_BASE*/, "Persistent kmap() Area" }, +-# endif +- { 0/*FIXADDR_START*/, "Fixmap Area" }, +-#endif +- { -1, NULL } /* End of list */ ++ [CPU_ENTRY_AREA_NR] = { 0UL, "CPU entry area" }, ++ [FIXADDR_START_NR] = { 0UL, "Fixmap area" }, ++ [END_OF_SPACE_NR] = { -1, NULL } + }; + ++#endif /* !CONFIG_X86_64 */ ++ + /* Multipliers for offsets within the PTEs */ + #define PTE_LEVEL_MULT (PAGE_SIZE) + #define PMD_LEVEL_MULT (PTRS_PER_PTE * PTE_LEVEL_MULT) +@@ -140,7 +160,7 @@ static void printk_prot(struct seq_file *m, pgprot_t prot, int level, bool dmsg) + static const char * const level_name[] = + { "cr3", "pgd", "p4d", "pud", "pmd", "pte" }; + +- if (!pgprot_val(prot)) { ++ if (!(pr & _PAGE_PRESENT)) { + /* Not present */ + pt_dump_cont_printf(m, dmsg, " "); + } else { +@@ -525,8 +545,8 @@ static int __init pt_dump_init(void) + address_markers[PKMAP_BASE_NR].start_address = PKMAP_BASE; + # endif + address_markers[FIXADDR_START_NR].start_address = FIXADDR_START; ++ address_markers[CPU_ENTRY_AREA_NR].start_address = CPU_ENTRY_AREA_BASE; + #endif +- + return 0; + } + __initcall(pt_dump_init); +diff --git a/arch/x86/mm/init_32.c b/arch/x86/mm/init_32.c +index 8a64a6f2848d..135c9a7898c7 100644 +--- a/arch/x86/mm/init_32.c ++++ b/arch/x86/mm/init_32.c +@@ -50,6 +50,7 @@ + #include + #include + #include ++#include + #include + + #include "mm_internal.h" +@@ -766,6 +767,7 @@ void __init mem_init(void) + mem_init_print_info(NULL); + printk(KERN_INFO "virtual kernel memory layout:\n" + " fixmap : 0x%08lx - 0x%08lx (%4ld kB)\n" ++ " cpu_entry : 0x%08lx - 0x%08lx (%4ld kB)\n" + #ifdef CONFIG_HIGHMEM + " pkmap : 0x%08lx - 0x%08lx (%4ld kB)\n" + #endif +@@ -777,6 +779,10 @@ void __init mem_init(void) + FIXADDR_START, FIXADDR_TOP, + (FIXADDR_TOP - FIXADDR_START) >> 10, + ++ CPU_ENTRY_AREA_BASE, ++ CPU_ENTRY_AREA_BASE + CPU_ENTRY_AREA_MAP_SIZE, ++ CPU_ENTRY_AREA_MAP_SIZE >> 10, ++ + #ifdef CONFIG_HIGHMEM + PKMAP_BASE, PKMAP_BASE+LAST_PKMAP*PAGE_SIZE, + (LAST_PKMAP*PAGE_SIZE) >> 10, +diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c +index 9ec70d780f1f..47388f0c0e59 100644 +--- a/arch/x86/mm/kasan_init_64.c ++++ b/arch/x86/mm/kasan_init_64.c +@@ -15,6 +15,7 @@ + #include + #include + #include ++#include + + extern struct range pfn_mapped[E820_MAX_ENTRIES]; + +@@ -322,31 +323,33 @@ void __init kasan_init(void) + map_range(&pfn_mapped[i]); + } + +- kasan_populate_zero_shadow( +- kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), +- kasan_mem_to_shadow((void *)__START_KERNEL_map)); +- +- kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext), +- (unsigned long)kasan_mem_to_shadow(_end), +- early_pfn_to_nid(__pa(_stext))); +- +- shadow_cpu_entry_begin = (void *)__fix_to_virt(FIX_CPU_ENTRY_AREA_BOTTOM); ++ shadow_cpu_entry_begin = (void *)CPU_ENTRY_AREA_BASE; + shadow_cpu_entry_begin = kasan_mem_to_shadow(shadow_cpu_entry_begin); + shadow_cpu_entry_begin = (void *)round_down((unsigned long)shadow_cpu_entry_begin, + PAGE_SIZE); + +- shadow_cpu_entry_end = (void *)(__fix_to_virt(FIX_CPU_ENTRY_AREA_TOP) + PAGE_SIZE); ++ shadow_cpu_entry_end = (void *)(CPU_ENTRY_AREA_BASE + ++ CPU_ENTRY_AREA_MAP_SIZE); + shadow_cpu_entry_end = kasan_mem_to_shadow(shadow_cpu_entry_end); + shadow_cpu_entry_end = (void *)round_up((unsigned long)shadow_cpu_entry_end, + PAGE_SIZE); + +- kasan_populate_zero_shadow(kasan_mem_to_shadow((void *)MODULES_END), +- shadow_cpu_entry_begin); ++ kasan_populate_zero_shadow( ++ kasan_mem_to_shadow((void *)PAGE_OFFSET + MAXMEM), ++ shadow_cpu_entry_begin); + + kasan_populate_shadow((unsigned long)shadow_cpu_entry_begin, + (unsigned long)shadow_cpu_entry_end, 0); + +- kasan_populate_zero_shadow(shadow_cpu_entry_end, (void *)KASAN_SHADOW_END); ++ kasan_populate_zero_shadow(shadow_cpu_entry_end, ++ kasan_mem_to_shadow((void *)__START_KERNEL_map)); ++ ++ kasan_populate_shadow((unsigned long)kasan_mem_to_shadow(_stext), ++ (unsigned long)kasan_mem_to_shadow(_end), ++ early_pfn_to_nid(__pa(_stext))); ++ ++ kasan_populate_zero_shadow(kasan_mem_to_shadow((void *)MODULES_END), ++ (void *)KASAN_SHADOW_END); + + load_cr3(init_top_pgt); + __flush_tlb_all(); +diff --git a/arch/x86/mm/pgtable_32.c b/arch/x86/mm/pgtable_32.c +index 6b9bf023a700..c3c5274410a9 100644 +--- a/arch/x86/mm/pgtable_32.c ++++ b/arch/x86/mm/pgtable_32.c +@@ -10,6 +10,7 @@ + #include + #include + ++#include + #include + #include + #include +diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c +index 3118392cdf75..0a1be3adc97e 100644 +--- a/arch/x86/mm/tlb.c ++++ b/arch/x86/mm/tlb.c +@@ -128,7 +128,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, + * isn't free. + */ + #ifdef CONFIG_DEBUG_VM +- if (WARN_ON_ONCE(__read_cr3() != build_cr3(real_prev, prev_asid))) { ++ if (WARN_ON_ONCE(__read_cr3() != build_cr3(real_prev->pgd, prev_asid))) { + /* + * If we were to BUG here, we'd be very likely to kill + * the system so hard that we don't see the call trace. +@@ -195,7 +195,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, + if (need_flush) { + this_cpu_write(cpu_tlbstate.ctxs[new_asid].ctx_id, next->context.ctx_id); + this_cpu_write(cpu_tlbstate.ctxs[new_asid].tlb_gen, next_tlb_gen); +- write_cr3(build_cr3(next, new_asid)); ++ write_cr3(build_cr3(next->pgd, new_asid)); + + /* + * NB: This gets called via leave_mm() in the idle path +@@ -208,7 +208,7 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next, + trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, TLB_FLUSH_ALL); + } else { + /* The new ASID is already up to date. */ +- write_cr3(build_cr3_noflush(next, new_asid)); ++ write_cr3(build_cr3_noflush(next->pgd, new_asid)); + + /* See above wrt _rcuidle. */ + trace_tlb_flush_rcuidle(TLB_FLUSH_ON_TASK_SWITCH, 0); +@@ -288,7 +288,7 @@ void initialize_tlbstate_and_flush(void) + !(cr4_read_shadow() & X86_CR4_PCIDE)); + + /* Force ASID 0 and force a TLB flush. */ +- write_cr3(build_cr3(mm, 0)); ++ write_cr3(build_cr3(mm->pgd, 0)); + + /* Reinitialize tlbstate. */ + this_cpu_write(cpu_tlbstate.loaded_mm_asid, 0); +@@ -551,7 +551,7 @@ static void do_kernel_range_flush(void *info) + + /* flush range by one by one 'invlpg' */ + for (addr = f->start; addr < f->end; addr += PAGE_SIZE) +- __flush_tlb_single(addr); ++ __flush_tlb_one(addr); + } + + void flush_tlb_kernel_range(unsigned long start, unsigned long end) +diff --git a/arch/x86/platform/uv/tlb_uv.c b/arch/x86/platform/uv/tlb_uv.c +index f44c0bc95aa2..8538a6723171 100644 +--- a/arch/x86/platform/uv/tlb_uv.c ++++ b/arch/x86/platform/uv/tlb_uv.c +@@ -299,7 +299,7 @@ static void bau_process_message(struct msg_desc *mdp, struct bau_control *bcp, + local_flush_tlb(); + stat->d_alltlb++; + } else { +- __flush_tlb_one(msg->address); ++ __flush_tlb_single(msg->address); + stat->d_onetlb++; + } + stat->d_requestee++; +diff --git a/arch/x86/xen/mmu_pv.c b/arch/x86/xen/mmu_pv.c +index c2454237fa67..a0e2b8c6e5c7 100644 +--- a/arch/x86/xen/mmu_pv.c ++++ b/arch/x86/xen/mmu_pv.c +@@ -2261,7 +2261,6 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) + + switch (idx) { + case FIX_BTMAP_END ... FIX_BTMAP_BEGIN: +- case FIX_RO_IDT: + #ifdef CONFIG_X86_32 + case FIX_WP_TEST: + # ifdef CONFIG_HIGHMEM +@@ -2272,7 +2271,6 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot) + #endif + case FIX_TEXT_POKE0: + case FIX_TEXT_POKE1: +- case FIX_CPU_ENTRY_AREA_TOP ... FIX_CPU_ENTRY_AREA_BOTTOM: + /* All local page mappings */ + pte = pfn_pte(phys, prot); + break; +diff --git a/block/bio.c b/block/bio.c +index 33fa6b4af312..7f978eac9a7a 100644 +--- a/block/bio.c ++++ b/block/bio.c +@@ -599,6 +599,8 @@ void __bio_clone_fast(struct bio *bio, struct bio *bio_src) + bio->bi_disk = bio_src->bi_disk; + bio->bi_partno = bio_src->bi_partno; + bio_set_flag(bio, BIO_CLONED); ++ if (bio_flagged(bio_src, BIO_THROTTLED)) ++ bio_set_flag(bio, BIO_THROTTLED); + bio->bi_opf = bio_src->bi_opf; + bio->bi_write_hint = bio_src->bi_write_hint; + bio->bi_iter = bio_src->bi_iter; +diff --git a/block/blk-throttle.c b/block/blk-throttle.c +index 8631763866c6..a8cd7b3d9647 100644 +--- a/block/blk-throttle.c ++++ b/block/blk-throttle.c +@@ -2223,13 +2223,7 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg, + out_unlock: + spin_unlock_irq(q->queue_lock); + out: +- /* +- * As multiple blk-throtls may stack in the same issue path, we +- * don't want bios to leave with the flag set. Clear the flag if +- * being issued. +- */ +- if (!throttled) +- bio_clear_flag(bio, BIO_THROTTLED); ++ bio_set_flag(bio, BIO_THROTTLED); + + #ifdef CONFIG_BLK_DEV_THROTTLING_LOW + if (throttled || !td->track_bio_latency) +diff --git a/crypto/af_alg.c b/crypto/af_alg.c +index e181073ef64d..6ec360213107 100644 +--- a/crypto/af_alg.c ++++ b/crypto/af_alg.c +@@ -1165,12 +1165,6 @@ int af_alg_get_rsgl(struct sock *sk, struct msghdr *msg, int flags, + if (!af_alg_readable(sk)) + break; + +- if (!ctx->used) { +- err = af_alg_wait_for_data(sk, flags); +- if (err) +- return err; +- } +- + seglen = min_t(size_t, (maxsize - len), + msg_data_left(msg)); + +diff --git a/crypto/algif_aead.c b/crypto/algif_aead.c +index 3d793bc2aa82..782cb8fec323 100644 +--- a/crypto/algif_aead.c ++++ b/crypto/algif_aead.c +@@ -111,6 +111,12 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, + size_t usedpages = 0; /* [in] RX bufs to be used from user */ + size_t processed = 0; /* [in] TX bufs to be consumed */ + ++ if (!ctx->used) { ++ err = af_alg_wait_for_data(sk, flags); ++ if (err) ++ return err; ++ } ++ + /* + * Data length provided by caller via sendmsg/sendpage that has not + * yet been processed. +@@ -285,6 +291,10 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, + /* AIO operation */ + sock_hold(sk); + areq->iocb = msg->msg_iocb; ++ ++ /* Remember output size that will be generated. */ ++ areq->outlen = outlen; ++ + aead_request_set_callback(&areq->cra_u.aead_req, + CRYPTO_TFM_REQ_MAY_BACKLOG, + af_alg_async_cb, areq); +@@ -292,12 +302,8 @@ static int _aead_recvmsg(struct socket *sock, struct msghdr *msg, + crypto_aead_decrypt(&areq->cra_u.aead_req); + + /* AIO operation in progress */ +- if (err == -EINPROGRESS || err == -EBUSY) { +- /* Remember output size that will be generated. */ +- areq->outlen = outlen; +- ++ if (err == -EINPROGRESS || err == -EBUSY) + return -EIOCBQUEUED; +- } + + sock_put(sk); + } else { +diff --git a/crypto/algif_skcipher.c b/crypto/algif_skcipher.c +index 30ee2a8e8f42..7a3e663d54d5 100644 +--- a/crypto/algif_skcipher.c ++++ b/crypto/algif_skcipher.c +@@ -72,6 +72,12 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg, + int err = 0; + size_t len = 0; + ++ if (!ctx->used) { ++ err = af_alg_wait_for_data(sk, flags); ++ if (err) ++ return err; ++ } ++ + /* Allocate cipher request for current operation. */ + areq = af_alg_alloc_areq(sk, sizeof(struct af_alg_async_req) + + crypto_skcipher_reqsize(tfm)); +@@ -119,6 +125,10 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg, + /* AIO operation */ + sock_hold(sk); + areq->iocb = msg->msg_iocb; ++ ++ /* Remember output size that will be generated. */ ++ areq->outlen = len; ++ + skcipher_request_set_callback(&areq->cra_u.skcipher_req, + CRYPTO_TFM_REQ_MAY_SLEEP, + af_alg_async_cb, areq); +@@ -127,12 +137,8 @@ static int _skcipher_recvmsg(struct socket *sock, struct msghdr *msg, + crypto_skcipher_decrypt(&areq->cra_u.skcipher_req); + + /* AIO operation in progress */ +- if (err == -EINPROGRESS || err == -EBUSY) { +- /* Remember output size that will be generated. */ +- areq->outlen = len; +- ++ if (err == -EINPROGRESS || err == -EBUSY) + return -EIOCBQUEUED; +- } + + sock_put(sk); + } else { +diff --git a/crypto/mcryptd.c b/crypto/mcryptd.c +index 4e6472658852..eca04d3729b3 100644 +--- a/crypto/mcryptd.c ++++ b/crypto/mcryptd.c +@@ -81,6 +81,7 @@ static int mcryptd_init_queue(struct mcryptd_queue *queue, + pr_debug("cpu_queue #%d %p\n", cpu, queue->cpu_queue); + crypto_init_queue(&cpu_queue->queue, max_cpu_qlen); + INIT_WORK(&cpu_queue->work, mcryptd_queue_worker); ++ spin_lock_init(&cpu_queue->q_lock); + } + return 0; + } +@@ -104,15 +105,16 @@ static int mcryptd_enqueue_request(struct mcryptd_queue *queue, + int cpu, err; + struct mcryptd_cpu_queue *cpu_queue; + +- cpu = get_cpu(); +- cpu_queue = this_cpu_ptr(queue->cpu_queue); +- rctx->tag.cpu = cpu; ++ cpu_queue = raw_cpu_ptr(queue->cpu_queue); ++ spin_lock(&cpu_queue->q_lock); ++ cpu = smp_processor_id(); ++ rctx->tag.cpu = smp_processor_id(); + + err = crypto_enqueue_request(&cpu_queue->queue, request); + pr_debug("enqueue request: cpu %d cpu_queue %p request %p\n", + cpu, cpu_queue, request); ++ spin_unlock(&cpu_queue->q_lock); + queue_work_on(cpu, kcrypto_wq, &cpu_queue->work); +- put_cpu(); + + return err; + } +@@ -161,16 +163,11 @@ static void mcryptd_queue_worker(struct work_struct *work) + cpu_queue = container_of(work, struct mcryptd_cpu_queue, work); + i = 0; + while (i < MCRYPTD_BATCH || single_task_running()) { +- /* +- * preempt_disable/enable is used to prevent +- * being preempted by mcryptd_enqueue_request() +- */ +- local_bh_disable(); +- preempt_disable(); ++ ++ spin_lock_bh(&cpu_queue->q_lock); + backlog = crypto_get_backlog(&cpu_queue->queue); + req = crypto_dequeue_request(&cpu_queue->queue); +- preempt_enable(); +- local_bh_enable(); ++ spin_unlock_bh(&cpu_queue->q_lock); + + if (!req) { + mcryptd_opportunistic_flush(); +@@ -185,7 +182,7 @@ static void mcryptd_queue_worker(struct work_struct *work) + ++i; + } + if (cpu_queue->queue.qlen) +- queue_work(kcrypto_wq, &cpu_queue->work); ++ queue_work_on(smp_processor_id(), kcrypto_wq, &cpu_queue->work); + } + + void mcryptd_flusher(struct work_struct *__work) +diff --git a/crypto/skcipher.c b/crypto/skcipher.c +index 778e0ff42bfa..11af5fd6a443 100644 +--- a/crypto/skcipher.c ++++ b/crypto/skcipher.c +@@ -449,6 +449,8 @@ static int skcipher_walk_skcipher(struct skcipher_walk *walk, + + walk->total = req->cryptlen; + walk->nbytes = 0; ++ walk->iv = req->iv; ++ walk->oiv = req->iv; + + if (unlikely(!walk->total)) + return 0; +@@ -456,9 +458,6 @@ static int skcipher_walk_skcipher(struct skcipher_walk *walk, + scatterwalk_start(&walk->in, req->src); + scatterwalk_start(&walk->out, req->dst); + +- walk->iv = req->iv; +- walk->oiv = req->iv; +- + walk->flags &= ~SKCIPHER_WALK_SLEEP; + walk->flags |= req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP ? + SKCIPHER_WALK_SLEEP : 0; +@@ -510,6 +509,8 @@ static int skcipher_walk_aead_common(struct skcipher_walk *walk, + int err; + + walk->nbytes = 0; ++ walk->iv = req->iv; ++ walk->oiv = req->iv; + + if (unlikely(!walk->total)) + return 0; +@@ -525,9 +526,6 @@ static int skcipher_walk_aead_common(struct skcipher_walk *walk, + scatterwalk_done(&walk->in, 0, walk->total); + scatterwalk_done(&walk->out, 0, walk->total); + +- walk->iv = req->iv; +- walk->oiv = req->iv; +- + if (req->base.flags & CRYPTO_TFM_REQ_MAY_SLEEP) + walk->flags |= SKCIPHER_WALK_SLEEP; + else +diff --git a/drivers/acpi/apei/erst.c b/drivers/acpi/apei/erst.c +index 2c462beee551..a943cf17faa7 100644 +--- a/drivers/acpi/apei/erst.c ++++ b/drivers/acpi/apei/erst.c +@@ -1007,7 +1007,7 @@ static ssize_t erst_reader(struct pstore_record *record) + /* The record may be cleared by others, try read next record */ + if (len == -ENOENT) + goto skip; +- else if (len < sizeof(*rcd)) { ++ else if (len < 0 || len < sizeof(*rcd)) { + rc = -EIO; + goto out; + } +diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c +index 9c2c49b6a240..dea0fb3d6f64 100644 +--- a/drivers/acpi/nfit/core.c ++++ b/drivers/acpi/nfit/core.c +@@ -1457,6 +1457,11 @@ static int acpi_nfit_add_dimm(struct acpi_nfit_desc *acpi_desc, + dev_name(&adev_dimm->dev)); + return -ENXIO; + } ++ /* ++ * Record nfit_mem for the notification path to track back to ++ * the nfit sysfs attributes for this dimm device object. ++ */ ++ dev_set_drvdata(&adev_dimm->dev, nfit_mem); + + /* + * Until standardization materializes we need to consider 4 +@@ -1516,9 +1521,11 @@ static void shutdown_dimm_notify(void *data) + sysfs_put(nfit_mem->flags_attr); + nfit_mem->flags_attr = NULL; + } +- if (adev_dimm) ++ if (adev_dimm) { + acpi_remove_notify_handler(adev_dimm->handle, + ACPI_DEVICE_NOTIFY, acpi_nvdimm_notify); ++ dev_set_drvdata(&adev_dimm->dev, NULL); ++ } + } + mutex_unlock(&acpi_desc->init_mutex); + } +diff --git a/drivers/char/ipmi/ipmi_si_intf.c b/drivers/char/ipmi/ipmi_si_intf.c +index e1cbb78c6806..c04aa11f0e21 100644 +--- a/drivers/char/ipmi/ipmi_si_intf.c ++++ b/drivers/char/ipmi/ipmi_si_intf.c +@@ -3469,7 +3469,6 @@ static int add_smi(struct smi_info *new_smi) + ipmi_addr_src_to_str(new_smi->addr_source), + si_to_str[new_smi->si_type]); + rv = -EBUSY; +- kfree(new_smi); + goto out_err; + } + } +diff --git a/drivers/clk/sunxi/clk-sun9i-mmc.c b/drivers/clk/sunxi/clk-sun9i-mmc.c +index 6041bdba2e97..f69f9e8c6f38 100644 +--- a/drivers/clk/sunxi/clk-sun9i-mmc.c ++++ b/drivers/clk/sunxi/clk-sun9i-mmc.c +@@ -16,6 +16,7 @@ + + #include + #include ++#include + #include + #include + #include +@@ -83,9 +84,20 @@ static int sun9i_mmc_reset_deassert(struct reset_controller_dev *rcdev, + return 0; + } + ++static int sun9i_mmc_reset_reset(struct reset_controller_dev *rcdev, ++ unsigned long id) ++{ ++ sun9i_mmc_reset_assert(rcdev, id); ++ udelay(10); ++ sun9i_mmc_reset_deassert(rcdev, id); ++ ++ return 0; ++} ++ + static const struct reset_control_ops sun9i_mmc_reset_ops = { + .assert = sun9i_mmc_reset_assert, + .deassert = sun9i_mmc_reset_deassert, ++ .reset = sun9i_mmc_reset_reset, + }; + + static int sun9i_a80_mmc_config_clk_probe(struct platform_device *pdev) +diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c +index dc1faa49687d..3b2c0538e48d 100644 +--- a/drivers/gpu/drm/i915/i915_gem.c ++++ b/drivers/gpu/drm/i915/i915_gem.c +@@ -325,17 +325,10 @@ int i915_gem_object_unbind(struct drm_i915_gem_object *obj) + * must wait for all rendering to complete to the object (as unbinding + * must anyway), and retire the requests. + */ +- ret = i915_gem_object_wait(obj, +- I915_WAIT_INTERRUPTIBLE | +- I915_WAIT_LOCKED | +- I915_WAIT_ALL, +- MAX_SCHEDULE_TIMEOUT, +- NULL); ++ ret = i915_gem_object_set_to_cpu_domain(obj, false); + if (ret) + return ret; + +- i915_gem_retire_requests(to_i915(obj->base.dev)); +- + while ((vma = list_first_entry_or_null(&obj->vma_list, + struct i915_vma, + obj_link))) { +diff --git a/drivers/gpu/drm/sun4i/sun4i_tcon.c b/drivers/gpu/drm/sun4i/sun4i_tcon.c +index d9791292553e..7b909d814d38 100644 +--- a/drivers/gpu/drm/sun4i/sun4i_tcon.c ++++ b/drivers/gpu/drm/sun4i/sun4i_tcon.c +@@ -567,12 +567,12 @@ static int sun4i_tcon_bind(struct device *dev, struct device *master, + if (IS_ERR(tcon->crtc)) { + dev_err(dev, "Couldn't create our CRTC\n"); + ret = PTR_ERR(tcon->crtc); +- goto err_free_clocks; ++ goto err_free_dotclock; + } + + ret = sun4i_rgb_init(drm, tcon); + if (ret < 0) +- goto err_free_clocks; ++ goto err_free_dotclock; + + list_add_tail(&tcon->list, &drv->tcon_list); + +diff --git a/drivers/mfd/cros_ec_spi.c b/drivers/mfd/cros_ec_spi.c +index c9714072e224..a14196e95e9b 100644 +--- a/drivers/mfd/cros_ec_spi.c ++++ b/drivers/mfd/cros_ec_spi.c +@@ -667,6 +667,7 @@ static int cros_ec_spi_probe(struct spi_device *spi) + sizeof(struct ec_response_get_protocol_info); + ec_dev->dout_size = sizeof(struct ec_host_request); + ++ ec_spi->last_transfer_ns = ktime_get_ns(); + + err = cros_ec_register(ec_dev); + if (err) { +diff --git a/drivers/mfd/twl4030-audio.c b/drivers/mfd/twl4030-audio.c +index da16bf45fab4..dc94ffc6321a 100644 +--- a/drivers/mfd/twl4030-audio.c ++++ b/drivers/mfd/twl4030-audio.c +@@ -159,13 +159,18 @@ unsigned int twl4030_audio_get_mclk(void) + EXPORT_SYMBOL_GPL(twl4030_audio_get_mclk); + + static bool twl4030_audio_has_codec(struct twl4030_audio_data *pdata, +- struct device_node *node) ++ struct device_node *parent) + { ++ struct device_node *node; ++ + if (pdata && pdata->codec) + return true; + +- if (of_find_node_by_name(node, "codec")) ++ node = of_get_child_by_name(parent, "codec"); ++ if (node) { ++ of_node_put(node); + return true; ++ } + + return false; + } +diff --git a/drivers/mfd/twl6040.c b/drivers/mfd/twl6040.c +index d66502d36ba0..dd19f17a1b63 100644 +--- a/drivers/mfd/twl6040.c ++++ b/drivers/mfd/twl6040.c +@@ -97,12 +97,16 @@ static struct reg_sequence twl6040_patch[] = { + }; + + +-static bool twl6040_has_vibra(struct device_node *node) ++static bool twl6040_has_vibra(struct device_node *parent) + { +-#ifdef CONFIG_OF +- if (of_find_node_by_name(node, "vibra")) ++ struct device_node *node; ++ ++ node = of_get_child_by_name(parent, "vibra"); ++ if (node) { ++ of_node_put(node); + return true; +-#endif ++ } ++ + return false; + } + +diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c +index bc93b69cfd1e..a539263cd79c 100644 +--- a/drivers/net/ethernet/marvell/mvneta.c ++++ b/drivers/net/ethernet/marvell/mvneta.c +@@ -1214,6 +1214,10 @@ static void mvneta_port_disable(struct mvneta_port *pp) + val &= ~MVNETA_GMAC0_PORT_ENABLE; + mvreg_write(pp, MVNETA_GMAC_CTRL_0, val); + ++ pp->link = 0; ++ pp->duplex = -1; ++ pp->speed = 0; ++ + udelay(200); + } + +@@ -1958,9 +1962,9 @@ static int mvneta_rx_swbm(struct mvneta_port *pp, int rx_todo, + + if (!mvneta_rxq_desc_is_first_last(rx_status) || + (rx_status & MVNETA_RXD_ERR_SUMMARY)) { ++ mvneta_rx_error(pp, rx_desc); + err_drop_frame: + dev->stats.rx_errors++; +- mvneta_rx_error(pp, rx_desc); + /* leave the descriptor untouched */ + continue; + } +@@ -3011,7 +3015,7 @@ static void mvneta_cleanup_rxqs(struct mvneta_port *pp) + { + int queue; + +- for (queue = 0; queue < txq_number; queue++) ++ for (queue = 0; queue < rxq_number; queue++) + mvneta_rxq_deinit(pp, &pp->rxqs[queue]); + } + +diff --git a/drivers/nvdimm/btt.c b/drivers/nvdimm/btt.c +index d5612bd1cc81..09428ebd315b 100644 +--- a/drivers/nvdimm/btt.c ++++ b/drivers/nvdimm/btt.c +@@ -210,12 +210,12 @@ static int btt_map_read(struct arena_info *arena, u32 lba, u32 *mapping, + return ret; + } + +-static int btt_log_read_pair(struct arena_info *arena, u32 lane, +- struct log_entry *ent) ++static int btt_log_group_read(struct arena_info *arena, u32 lane, ++ struct log_group *log) + { + return arena_read_bytes(arena, +- arena->logoff + (2 * lane * LOG_ENT_SIZE), ent, +- 2 * LOG_ENT_SIZE, 0); ++ arena->logoff + (lane * LOG_GRP_SIZE), log, ++ LOG_GRP_SIZE, 0); + } + + static struct dentry *debugfs_root; +@@ -255,6 +255,8 @@ static void arena_debugfs_init(struct arena_info *a, struct dentry *parent, + debugfs_create_x64("logoff", S_IRUGO, d, &a->logoff); + debugfs_create_x64("info2off", S_IRUGO, d, &a->info2off); + debugfs_create_x32("flags", S_IRUGO, d, &a->flags); ++ debugfs_create_u32("log_index_0", S_IRUGO, d, &a->log_index[0]); ++ debugfs_create_u32("log_index_1", S_IRUGO, d, &a->log_index[1]); + } + + static void btt_debugfs_init(struct btt *btt) +@@ -273,6 +275,11 @@ static void btt_debugfs_init(struct btt *btt) + } + } + ++static u32 log_seq(struct log_group *log, int log_idx) ++{ ++ return le32_to_cpu(log->ent[log_idx].seq); ++} ++ + /* + * This function accepts two log entries, and uses the + * sequence number to find the 'older' entry. +@@ -282,8 +289,10 @@ static void btt_debugfs_init(struct btt *btt) + * + * TODO The logic feels a bit kludge-y. make it better.. + */ +-static int btt_log_get_old(struct log_entry *ent) ++static int btt_log_get_old(struct arena_info *a, struct log_group *log) + { ++ int idx0 = a->log_index[0]; ++ int idx1 = a->log_index[1]; + int old; + + /* +@@ -291,23 +300,23 @@ static int btt_log_get_old(struct log_entry *ent) + * the next time, the following logic works out to put this + * (next) entry into [1] + */ +- if (ent[0].seq == 0) { +- ent[0].seq = cpu_to_le32(1); ++ if (log_seq(log, idx0) == 0) { ++ log->ent[idx0].seq = cpu_to_le32(1); + return 0; + } + +- if (ent[0].seq == ent[1].seq) ++ if (log_seq(log, idx0) == log_seq(log, idx1)) + return -EINVAL; +- if (le32_to_cpu(ent[0].seq) + le32_to_cpu(ent[1].seq) > 5) ++ if (log_seq(log, idx0) + log_seq(log, idx1) > 5) + return -EINVAL; + +- if (le32_to_cpu(ent[0].seq) < le32_to_cpu(ent[1].seq)) { +- if (le32_to_cpu(ent[1].seq) - le32_to_cpu(ent[0].seq) == 1) ++ if (log_seq(log, idx0) < log_seq(log, idx1)) { ++ if ((log_seq(log, idx1) - log_seq(log, idx0)) == 1) + old = 0; + else + old = 1; + } else { +- if (le32_to_cpu(ent[0].seq) - le32_to_cpu(ent[1].seq) == 1) ++ if ((log_seq(log, idx0) - log_seq(log, idx1)) == 1) + old = 1; + else + old = 0; +@@ -327,17 +336,18 @@ static int btt_log_read(struct arena_info *arena, u32 lane, + { + int ret; + int old_ent, ret_ent; +- struct log_entry log[2]; ++ struct log_group log; + +- ret = btt_log_read_pair(arena, lane, log); ++ ret = btt_log_group_read(arena, lane, &log); + if (ret) + return -EIO; + +- old_ent = btt_log_get_old(log); ++ old_ent = btt_log_get_old(arena, &log); + if (old_ent < 0 || old_ent > 1) { + dev_err(to_dev(arena), + "log corruption (%d): lane %d seq [%d, %d]\n", +- old_ent, lane, log[0].seq, log[1].seq); ++ old_ent, lane, log.ent[arena->log_index[0]].seq, ++ log.ent[arena->log_index[1]].seq); + /* TODO set error state? */ + return -EIO; + } +@@ -345,7 +355,7 @@ static int btt_log_read(struct arena_info *arena, u32 lane, + ret_ent = (old_flag ? old_ent : (1 - old_ent)); + + if (ent != NULL) +- memcpy(ent, &log[ret_ent], LOG_ENT_SIZE); ++ memcpy(ent, &log.ent[arena->log_index[ret_ent]], LOG_ENT_SIZE); + + return ret_ent; + } +@@ -359,17 +369,13 @@ static int __btt_log_write(struct arena_info *arena, u32 lane, + u32 sub, struct log_entry *ent, unsigned long flags) + { + int ret; +- /* +- * Ignore the padding in log_entry for calculating log_half. +- * The entry is 'committed' when we write the sequence number, +- * and we want to ensure that that is the last thing written. +- * We don't bother writing the padding as that would be extra +- * media wear and write amplification +- */ +- unsigned int log_half = (LOG_ENT_SIZE - 2 * sizeof(u64)) / 2; +- u64 ns_off = arena->logoff + (((2 * lane) + sub) * LOG_ENT_SIZE); ++ u32 group_slot = arena->log_index[sub]; ++ unsigned int log_half = LOG_ENT_SIZE / 2; + void *src = ent; ++ u64 ns_off; + ++ ns_off = arena->logoff + (lane * LOG_GRP_SIZE) + ++ (group_slot * LOG_ENT_SIZE); + /* split the 16B write into atomic, durable halves */ + ret = arena_write_bytes(arena, ns_off, src, log_half, flags); + if (ret) +@@ -452,7 +458,7 @@ static int btt_log_init(struct arena_info *arena) + { + size_t logsize = arena->info2off - arena->logoff; + size_t chunk_size = SZ_4K, offset = 0; +- struct log_entry log; ++ struct log_entry ent; + void *zerobuf; + int ret; + u32 i; +@@ -484,11 +490,11 @@ static int btt_log_init(struct arena_info *arena) + } + + for (i = 0; i < arena->nfree; i++) { +- log.lba = cpu_to_le32(i); +- log.old_map = cpu_to_le32(arena->external_nlba + i); +- log.new_map = cpu_to_le32(arena->external_nlba + i); +- log.seq = cpu_to_le32(LOG_SEQ_INIT); +- ret = __btt_log_write(arena, i, 0, &log, 0); ++ ent.lba = cpu_to_le32(i); ++ ent.old_map = cpu_to_le32(arena->external_nlba + i); ++ ent.new_map = cpu_to_le32(arena->external_nlba + i); ++ ent.seq = cpu_to_le32(LOG_SEQ_INIT); ++ ret = __btt_log_write(arena, i, 0, &ent, 0); + if (ret) + goto free; + } +@@ -593,6 +599,123 @@ static int btt_freelist_init(struct arena_info *arena) + return 0; + } + ++static bool ent_is_padding(struct log_entry *ent) ++{ ++ return (ent->lba == 0) && (ent->old_map == 0) && (ent->new_map == 0) ++ && (ent->seq == 0); ++} ++ ++/* ++ * Detecting valid log indices: We read a log group (see the comments in btt.h ++ * for a description of a 'log_group' and its 'slots'), and iterate over its ++ * four slots. We expect that a padding slot will be all-zeroes, and use this ++ * to detect a padding slot vs. an actual entry. ++ * ++ * If a log_group is in the initial state, i.e. hasn't been used since the ++ * creation of this BTT layout, it will have three of the four slots with ++ * zeroes. We skip over these log_groups for the detection of log_index. If ++ * all log_groups are in the initial state (i.e. the BTT has never been ++ * written to), it is safe to assume the 'new format' of log entries in slots ++ * (0, 1). ++ */ ++static int log_set_indices(struct arena_info *arena) ++{ ++ bool idx_set = false, initial_state = true; ++ int ret, log_index[2] = {-1, -1}; ++ u32 i, j, next_idx = 0; ++ struct log_group log; ++ u32 pad_count = 0; ++ ++ for (i = 0; i < arena->nfree; i++) { ++ ret = btt_log_group_read(arena, i, &log); ++ if (ret < 0) ++ return ret; ++ ++ for (j = 0; j < 4; j++) { ++ if (!idx_set) { ++ if (ent_is_padding(&log.ent[j])) { ++ pad_count++; ++ continue; ++ } else { ++ /* Skip if index has been recorded */ ++ if ((next_idx == 1) && ++ (j == log_index[0])) ++ continue; ++ /* valid entry, record index */ ++ log_index[next_idx] = j; ++ next_idx++; ++ } ++ if (next_idx == 2) { ++ /* two valid entries found */ ++ idx_set = true; ++ } else if (next_idx > 2) { ++ /* too many valid indices */ ++ return -ENXIO; ++ } ++ } else { ++ /* ++ * once the indices have been set, just verify ++ * that all subsequent log groups are either in ++ * their initial state or follow the same ++ * indices. ++ */ ++ if (j == log_index[0]) { ++ /* entry must be 'valid' */ ++ if (ent_is_padding(&log.ent[j])) ++ return -ENXIO; ++ } else if (j == log_index[1]) { ++ ; ++ /* ++ * log_index[1] can be padding if the ++ * lane never got used and it is still ++ * in the initial state (three 'padding' ++ * entries) ++ */ ++ } else { ++ /* entry must be invalid (padding) */ ++ if (!ent_is_padding(&log.ent[j])) ++ return -ENXIO; ++ } ++ } ++ } ++ /* ++ * If any of the log_groups have more than one valid, ++ * non-padding entry, then the we are no longer in the ++ * initial_state ++ */ ++ if (pad_count < 3) ++ initial_state = false; ++ pad_count = 0; ++ } ++ ++ if (!initial_state && !idx_set) ++ return -ENXIO; ++ ++ /* ++ * If all the entries in the log were in the initial state, ++ * assume new padding scheme ++ */ ++ if (initial_state) ++ log_index[1] = 1; ++ ++ /* ++ * Only allow the known permutations of log/padding indices, ++ * i.e. (0, 1), and (0, 2) ++ */ ++ if ((log_index[0] == 0) && ((log_index[1] == 1) || (log_index[1] == 2))) ++ ; /* known index possibilities */ ++ else { ++ dev_err(to_dev(arena), "Found an unknown padding scheme\n"); ++ return -ENXIO; ++ } ++ ++ arena->log_index[0] = log_index[0]; ++ arena->log_index[1] = log_index[1]; ++ dev_dbg(to_dev(arena), "log_index_0 = %d\n", log_index[0]); ++ dev_dbg(to_dev(arena), "log_index_1 = %d\n", log_index[1]); ++ return 0; ++} ++ + static int btt_rtt_init(struct arena_info *arena) + { + arena->rtt = kcalloc(arena->nfree, sizeof(u32), GFP_KERNEL); +@@ -649,8 +772,7 @@ static struct arena_info *alloc_arena(struct btt *btt, size_t size, + available -= 2 * BTT_PG_SIZE; + + /* The log takes a fixed amount of space based on nfree */ +- logsize = roundup(2 * arena->nfree * sizeof(struct log_entry), +- BTT_PG_SIZE); ++ logsize = roundup(arena->nfree * LOG_GRP_SIZE, BTT_PG_SIZE); + available -= logsize; + + /* Calculate optimal split between map and data area */ +@@ -667,6 +789,10 @@ static struct arena_info *alloc_arena(struct btt *btt, size_t size, + arena->mapoff = arena->dataoff + datasize; + arena->logoff = arena->mapoff + mapsize; + arena->info2off = arena->logoff + logsize; ++ ++ /* Default log indices are (0,1) */ ++ arena->log_index[0] = 0; ++ arena->log_index[1] = 1; + return arena; + } + +@@ -757,6 +883,13 @@ static int discover_arenas(struct btt *btt) + arena->external_lba_start = cur_nlba; + parse_arena_meta(arena, super, cur_off); + ++ ret = log_set_indices(arena); ++ if (ret) { ++ dev_err(to_dev(arena), ++ "Unable to deduce log/padding indices\n"); ++ goto out; ++ } ++ + mutex_init(&arena->err_lock); + ret = btt_freelist_init(arena); + if (ret) +diff --git a/drivers/nvdimm/btt.h b/drivers/nvdimm/btt.h +index 578c2057524d..2609683c4167 100644 +--- a/drivers/nvdimm/btt.h ++++ b/drivers/nvdimm/btt.h +@@ -27,6 +27,7 @@ + #define MAP_ERR_MASK (1 << MAP_ERR_SHIFT) + #define MAP_LBA_MASK (~((1 << MAP_TRIM_SHIFT) | (1 << MAP_ERR_SHIFT))) + #define MAP_ENT_NORMAL 0xC0000000 ++#define LOG_GRP_SIZE sizeof(struct log_group) + #define LOG_ENT_SIZE sizeof(struct log_entry) + #define ARENA_MIN_SIZE (1UL << 24) /* 16 MB */ + #define ARENA_MAX_SIZE (1ULL << 39) /* 512 GB */ +@@ -50,12 +51,52 @@ enum btt_init_state { + INIT_READY + }; + ++/* ++ * A log group represents one log 'lane', and consists of four log entries. ++ * Two of the four entries are valid entries, and the remaining two are ++ * padding. Due to an old bug in the padding location, we need to perform a ++ * test to determine the padding scheme being used, and use that scheme ++ * thereafter. ++ * ++ * In kernels prior to 4.15, 'log group' would have actual log entries at ++ * indices (0, 2) and padding at indices (1, 3), where as the correct/updated ++ * format has log entries at indices (0, 1) and padding at indices (2, 3). ++ * ++ * Old (pre 4.15) format: ++ * +-----------------+-----------------+ ++ * | ent[0] | ent[1] | ++ * | 16B | 16B | ++ * | lba/old/new/seq | pad | ++ * +-----------------------------------+ ++ * | ent[2] | ent[3] | ++ * | 16B | 16B | ++ * | lba/old/new/seq | pad | ++ * +-----------------+-----------------+ ++ * ++ * New format: ++ * +-----------------+-----------------+ ++ * | ent[0] | ent[1] | ++ * | 16B | 16B | ++ * | lba/old/new/seq | lba/old/new/seq | ++ * +-----------------------------------+ ++ * | ent[2] | ent[3] | ++ * | 16B | 16B | ++ * | pad | pad | ++ * +-----------------+-----------------+ ++ * ++ * We detect during start-up which format is in use, and set ++ * arena->log_index[(0, 1)] with the detected format. ++ */ ++ + struct log_entry { + __le32 lba; + __le32 old_map; + __le32 new_map; + __le32 seq; +- __le64 padding[2]; ++}; ++ ++struct log_group { ++ struct log_entry ent[4]; + }; + + struct btt_sb { +@@ -125,6 +166,7 @@ struct aligned_lock { + * @list: List head for list of arenas + * @debugfs_dir: Debugfs dentry + * @flags: Arena flags - may signify error states. ++ * @log_index: Indices of the valid log entries in a log_group + * + * arena_info is a per-arena handle. Once an arena is narrowed down for an + * IO, this struct is passed around for the duration of the IO. +@@ -157,6 +199,7 @@ struct arena_info { + /* Arena flags */ + u32 flags; + struct mutex err_lock; ++ int log_index[2]; + }; + + /** +diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c +index 65cc171c721d..2adada1a5855 100644 +--- a/drivers/nvdimm/pfn_devs.c ++++ b/drivers/nvdimm/pfn_devs.c +@@ -364,9 +364,9 @@ struct device *nd_pfn_create(struct nd_region *nd_region) + int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig) + { + u64 checksum, offset; +- unsigned long align; + enum nd_pfn_mode mode; + struct nd_namespace_io *nsio; ++ unsigned long align, start_pad; + struct nd_pfn_sb *pfn_sb = nd_pfn->pfn_sb; + struct nd_namespace_common *ndns = nd_pfn->ndns; + const u8 *parent_uuid = nd_dev_to_uuid(&ndns->dev); +@@ -410,6 +410,7 @@ int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig) + + align = le32_to_cpu(pfn_sb->align); + offset = le64_to_cpu(pfn_sb->dataoff); ++ start_pad = le32_to_cpu(pfn_sb->start_pad); + if (align == 0) + align = 1UL << ilog2(offset); + mode = le32_to_cpu(pfn_sb->mode); +@@ -468,7 +469,7 @@ int nd_pfn_validate(struct nd_pfn *nd_pfn, const char *sig) + return -EBUSY; + } + +- if ((align && !IS_ALIGNED(offset, align)) ++ if ((align && !IS_ALIGNED(nsio->res.start + offset + start_pad, align)) + || !IS_ALIGNED(offset, PAGE_SIZE)) { + dev_err(&nd_pfn->dev, + "bad offset: %#llx dax disabled align: %#lx\n", +@@ -582,6 +583,12 @@ static struct vmem_altmap *__nvdimm_setup_pfn(struct nd_pfn *nd_pfn, + return altmap; + } + ++static u64 phys_pmem_align_down(struct nd_pfn *nd_pfn, u64 phys) ++{ ++ return min_t(u64, PHYS_SECTION_ALIGN_DOWN(phys), ++ ALIGN_DOWN(phys, nd_pfn->align)); ++} ++ + static int nd_pfn_init(struct nd_pfn *nd_pfn) + { + u32 dax_label_reserve = is_nd_dax(&nd_pfn->dev) ? SZ_128K : 0; +@@ -637,13 +644,16 @@ static int nd_pfn_init(struct nd_pfn *nd_pfn) + start = nsio->res.start; + size = PHYS_SECTION_ALIGN_UP(start + size) - start; + if (region_intersects(start, size, IORESOURCE_SYSTEM_RAM, +- IORES_DESC_NONE) == REGION_MIXED) { ++ IORES_DESC_NONE) == REGION_MIXED ++ || !IS_ALIGNED(start + resource_size(&nsio->res), ++ nd_pfn->align)) { + size = resource_size(&nsio->res); +- end_trunc = start + size - PHYS_SECTION_ALIGN_DOWN(start + size); ++ end_trunc = start + size - phys_pmem_align_down(nd_pfn, ++ start + size); + } + + if (start_pad + end_trunc) +- dev_info(&nd_pfn->dev, "%s section collision, truncate %d bytes\n", ++ dev_info(&nd_pfn->dev, "%s alignment collision, truncate %d bytes\n", + dev_name(&ndns->dev), start_pad + end_trunc); + + /* +diff --git a/drivers/parisc/lba_pci.c b/drivers/parisc/lba_pci.c +index a25fed52f7e9..41b740aed3a3 100644 +--- a/drivers/parisc/lba_pci.c ++++ b/drivers/parisc/lba_pci.c +@@ -1692,3 +1692,36 @@ void lba_set_iregs(struct parisc_device *lba, u32 ibase, u32 imask) + iounmap(base_addr); + } + ++ ++/* ++ * The design of the Diva management card in rp34x0 machines (rp3410, rp3440) ++ * seems rushed, so that many built-in components simply don't work. ++ * The following quirks disable the serial AUX port and the built-in ATI RV100 ++ * Radeon 7000 graphics card which both don't have any external connectors and ++ * thus are useless, and even worse, e.g. the AUX port occupies ttyS0 and as ++ * such makes those machines the only PARISC machines on which we can't use ++ * ttyS0 as boot console. ++ */ ++static void quirk_diva_ati_card(struct pci_dev *dev) ++{ ++ if (dev->subsystem_vendor != PCI_VENDOR_ID_HP || ++ dev->subsystem_device != 0x1292) ++ return; ++ ++ dev_info(&dev->dev, "Hiding Diva built-in ATI card"); ++ dev->device = 0; ++} ++DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_ATI, PCI_DEVICE_ID_ATI_RADEON_QY, ++ quirk_diva_ati_card); ++ ++static void quirk_diva_aux_disable(struct pci_dev *dev) ++{ ++ if (dev->subsystem_vendor != PCI_VENDOR_ID_HP || ++ dev->subsystem_device != 0x1291) ++ return; ++ ++ dev_info(&dev->dev, "Hiding Diva built-in AUX serial device"); ++ dev->device = 0; ++} ++DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_HP, PCI_DEVICE_ID_HP_DIVA_AUX, ++ quirk_diva_aux_disable); +diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c +index 11bd267fc137..bb0927de79dd 100644 +--- a/drivers/pci/pci-driver.c ++++ b/drivers/pci/pci-driver.c +@@ -968,7 +968,12 @@ static int pci_pm_thaw_noirq(struct device *dev) + if (pci_has_legacy_pm_support(pci_dev)) + return pci_legacy_resume_early(dev); + +- pci_update_current_state(pci_dev, PCI_D0); ++ /* ++ * pci_restore_state() requires the device to be in D0 (because of MSI ++ * restoration among other things), so force it into D0 in case the ++ * driver's "freeze" callbacks put it into a low-power state directly. ++ */ ++ pci_set_power_state(pci_dev, PCI_D0); + pci_restore_state(pci_dev); + + if (drv && drv->pm && drv->pm->thaw_noirq) +diff --git a/drivers/pinctrl/intel/pinctrl-cherryview.c b/drivers/pinctrl/intel/pinctrl-cherryview.c +index fadbca907c7c..0907531a02ca 100644 +--- a/drivers/pinctrl/intel/pinctrl-cherryview.c ++++ b/drivers/pinctrl/intel/pinctrl-cherryview.c +@@ -1620,6 +1620,22 @@ static int chv_gpio_probe(struct chv_pinctrl *pctrl, int irq) + clear_bit(i, chip->irq_valid_mask); + } + ++ /* ++ * The same set of machines in chv_no_valid_mask[] have incorrectly ++ * configured GPIOs that generate spurious interrupts so we use ++ * this same list to apply another quirk for them. ++ * ++ * See also https://bugzilla.kernel.org/show_bug.cgi?id=197953. ++ */ ++ if (!need_valid_mask) { ++ /* ++ * Mask all interrupts the community is able to generate ++ * but leave the ones that can only generate GPEs unmasked. ++ */ ++ chv_writel(GENMASK(31, pctrl->community->nirqs), ++ pctrl->regs + CHV_INTMASK); ++ } ++ + /* Clear all interrupts */ + chv_writel(0xffff, pctrl->regs + CHV_INTSTAT); + +diff --git a/drivers/spi/spi-armada-3700.c b/drivers/spi/spi-armada-3700.c +index 568e1c65aa82..fe3fa1e8517a 100644 +--- a/drivers/spi/spi-armada-3700.c ++++ b/drivers/spi/spi-armada-3700.c +@@ -79,6 +79,7 @@ + #define A3700_SPI_BYTE_LEN BIT(5) + #define A3700_SPI_CLK_PRESCALE BIT(0) + #define A3700_SPI_CLK_PRESCALE_MASK (0x1f) ++#define A3700_SPI_CLK_EVEN_OFFS (0x10) + + #define A3700_SPI_WFIFO_THRS_BIT 28 + #define A3700_SPI_RFIFO_THRS_BIT 24 +@@ -220,6 +221,13 @@ static void a3700_spi_clock_set(struct a3700_spi *a3700_spi, + + prescale = DIV_ROUND_UP(clk_get_rate(a3700_spi->clk), speed_hz); + ++ /* For prescaler values over 15, we can only set it by steps of 2. ++ * Starting from A3700_SPI_CLK_EVEN_OFFS, we set values from 0 up to ++ * 30. We only use this range from 16 to 30. ++ */ ++ if (prescale > 15) ++ prescale = A3700_SPI_CLK_EVEN_OFFS + DIV_ROUND_UP(prescale, 2); ++ + val = spireg_read(a3700_spi, A3700_SPI_IF_CFG_REG); + val = val & ~A3700_SPI_CLK_PRESCALE_MASK; + +diff --git a/drivers/spi/spi-xilinx.c b/drivers/spi/spi-xilinx.c +index bc7100b93dfc..e0b9fe1d0e37 100644 +--- a/drivers/spi/spi-xilinx.c ++++ b/drivers/spi/spi-xilinx.c +@@ -271,6 +271,7 @@ static int xilinx_spi_txrx_bufs(struct spi_device *spi, struct spi_transfer *t) + while (remaining_words) { + int n_words, tx_words, rx_words; + u32 sr; ++ int stalled; + + n_words = min(remaining_words, xspi->buffer_size); + +@@ -299,7 +300,17 @@ static int xilinx_spi_txrx_bufs(struct spi_device *spi, struct spi_transfer *t) + + /* Read out all the data from the Rx FIFO */ + rx_words = n_words; ++ stalled = 10; + while (rx_words) { ++ if (rx_words == n_words && !(stalled--) && ++ !(sr & XSPI_SR_TX_EMPTY_MASK) && ++ (sr & XSPI_SR_RX_EMPTY_MASK)) { ++ dev_err(&spi->dev, ++ "Detected stall. Check C_SPI_MODE and C_SPI_MEMORY\n"); ++ xspi_init_hw(xspi); ++ return -EIO; ++ } ++ + if ((sr & XSPI_SR_TX_EMPTY_MASK) && (rx_words > 1)) { + xilinx_spi_rx(xspi); + rx_words--; +diff --git a/include/asm-generic/mm_hooks.h b/include/asm-generic/mm_hooks.h +index ea189d88a3cc..8ac4e68a12f0 100644 +--- a/include/asm-generic/mm_hooks.h ++++ b/include/asm-generic/mm_hooks.h +@@ -7,9 +7,10 @@ + #ifndef _ASM_GENERIC_MM_HOOKS_H + #define _ASM_GENERIC_MM_HOOKS_H + +-static inline void arch_dup_mmap(struct mm_struct *oldmm, +- struct mm_struct *mm) ++static inline int arch_dup_mmap(struct mm_struct *oldmm, ++ struct mm_struct *mm) + { ++ return 0; + } + + static inline void arch_exit_mmap(struct mm_struct *mm) +diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h +index 1ac457511f4e..045a7f52ab3a 100644 +--- a/include/asm-generic/pgtable.h ++++ b/include/asm-generic/pgtable.h +@@ -1025,6 +1025,11 @@ static inline int pmd_clear_huge(pmd_t *pmd) + struct file; + int phys_mem_access_prot_allowed(struct file *file, unsigned long pfn, + unsigned long size, pgprot_t *vma_prot); ++ ++#ifndef CONFIG_X86_ESPFIX64 ++static inline void init_espfix_bsp(void) { } ++#endif ++ + #endif /* !__ASSEMBLY__ */ + + #ifndef io_remap_pfn_range +diff --git a/include/crypto/mcryptd.h b/include/crypto/mcryptd.h +index cceafa01f907..b67404fc4b34 100644 +--- a/include/crypto/mcryptd.h ++++ b/include/crypto/mcryptd.h +@@ -27,6 +27,7 @@ static inline struct mcryptd_ahash *__mcryptd_ahash_cast( + + struct mcryptd_cpu_queue { + struct crypto_queue queue; ++ spinlock_t q_lock; + struct work_struct work; + }; + +diff --git a/include/linux/bio.h b/include/linux/bio.h +index 275c91c99516..45f00dd6323c 100644 +--- a/include/linux/bio.h ++++ b/include/linux/bio.h +@@ -504,6 +504,8 @@ extern unsigned int bvec_nr_vecs(unsigned short idx); + + #define bio_set_dev(bio, bdev) \ + do { \ ++ if ((bio)->bi_disk != (bdev)->bd_disk) \ ++ bio_clear_flag(bio, BIO_THROTTLED);\ + (bio)->bi_disk = (bdev)->bd_disk; \ + (bio)->bi_partno = (bdev)->bd_partno; \ + } while (0) +diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h +index 96ac3815542c..1c8a8a2aedf7 100644 +--- a/include/linux/blk_types.h ++++ b/include/linux/blk_types.h +@@ -50,8 +50,6 @@ struct blk_issue_stat { + struct bio { + struct bio *bi_next; /* request queue link */ + struct gendisk *bi_disk; +- u8 bi_partno; +- blk_status_t bi_status; + unsigned int bi_opf; /* bottom bits req flags, + * top bits REQ_OP. Use + * accessors. +@@ -59,8 +57,8 @@ struct bio { + unsigned short bi_flags; /* status, etc and bvec pool number */ + unsigned short bi_ioprio; + unsigned short bi_write_hint; +- +- struct bvec_iter bi_iter; ++ blk_status_t bi_status; ++ u8 bi_partno; + + /* Number of segments in this BIO after + * physical address coalescing is performed. +@@ -74,8 +72,9 @@ struct bio { + unsigned int bi_seg_front_size; + unsigned int bi_seg_back_size; + +- atomic_t __bi_remaining; ++ struct bvec_iter bi_iter; + ++ atomic_t __bi_remaining; + bio_end_io_t *bi_end_io; + + void *bi_private; +diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h +index 8da66379f7ea..fd47bd96b5d3 100644 +--- a/include/linux/blkdev.h ++++ b/include/linux/blkdev.h +@@ -135,7 +135,7 @@ typedef __u32 __bitwise req_flags_t; + struct request { + struct list_head queuelist; + union { +- call_single_data_t csd; ++ struct __call_single_data csd; + u64 fifo_time; + }; + +diff --git a/init/main.c b/init/main.c +index 0ee9c6866ada..8a390f60ec81 100644 +--- a/init/main.c ++++ b/init/main.c +@@ -504,6 +504,8 @@ static void __init mm_init(void) + pgtable_init(); + vmalloc_init(); + ioremap_huge_init(); ++ /* Should be run before the first non-init thread is created */ ++ init_espfix_bsp(); + } + + asmlinkage __visible void __init start_kernel(void) +@@ -673,10 +675,6 @@ asmlinkage __visible void __init start_kernel(void) + #ifdef CONFIG_X86 + if (efi_enabled(EFI_RUNTIME_SERVICES)) + efi_enter_virtual_mode(); +-#endif +-#ifdef CONFIG_X86_ESPFIX64 +- /* Should be run before the first non-init thread is created */ +- init_espfix_bsp(); + #endif + thread_stack_cache_init(); + cred_init(); +diff --git a/kernel/fork.c b/kernel/fork.c +index 07cc743698d3..500ce64517d9 100644 +--- a/kernel/fork.c ++++ b/kernel/fork.c +@@ -721,8 +721,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, + goto out; + } + /* a new mm has just been created */ +- arch_dup_mmap(oldmm, mm); +- retval = 0; ++ retval = arch_dup_mmap(oldmm, mm); + out: + up_write(&mm->mmap_sem); + flush_tlb_mm(oldmm); +diff --git a/net/ipv6/route.c b/net/ipv6/route.c +index 76b47682f77f..598efa8cfe25 100644 +--- a/net/ipv6/route.c ++++ b/net/ipv6/route.c +@@ -1055,6 +1055,7 @@ static struct rt6_info *rt6_get_pcpu_route(struct rt6_info *rt) + + static struct rt6_info *rt6_make_pcpu_route(struct rt6_info *rt) + { ++ struct fib6_table *table = rt->rt6i_table; + struct rt6_info *pcpu_rt, *prev, **p; + + pcpu_rt = ip6_rt_pcpu_alloc(rt); +@@ -1065,20 +1066,28 @@ static struct rt6_info *rt6_make_pcpu_route(struct rt6_info *rt) + return net->ipv6.ip6_null_entry; + } + +- dst_hold(&pcpu_rt->dst); +- p = this_cpu_ptr(rt->rt6i_pcpu); +- prev = cmpxchg(p, NULL, pcpu_rt); +- if (prev) { +- /* If someone did it before us, return prev instead */ +- /* release refcnt taken by ip6_rt_pcpu_alloc() */ +- dst_release_immediate(&pcpu_rt->dst); +- /* release refcnt taken by above dst_hold() */ ++ read_lock_bh(&table->tb6_lock); ++ if (rt->rt6i_pcpu) { ++ p = this_cpu_ptr(rt->rt6i_pcpu); ++ prev = cmpxchg(p, NULL, pcpu_rt); ++ if (prev) { ++ /* If someone did it before us, return prev instead */ ++ dst_release_immediate(&pcpu_rt->dst); ++ pcpu_rt = prev; ++ } ++ } else { ++ /* rt has been removed from the fib6 tree ++ * before we have a chance to acquire the read_lock. ++ * In this case, don't brother to create a pcpu rt ++ * since rt is going away anyway. The next ++ * dst_check() will trigger a re-lookup. ++ */ + dst_release_immediate(&pcpu_rt->dst); +- dst_hold(&prev->dst); +- pcpu_rt = prev; ++ pcpu_rt = rt; + } +- ++ dst_hold(&pcpu_rt->dst); + rt6_dst_from_metrics_check(pcpu_rt); ++ read_unlock_bh(&table->tb6_lock); + return pcpu_rt; + } + +@@ -1168,28 +1177,19 @@ struct rt6_info *ip6_pol_route(struct net *net, struct fib6_table *table, + if (pcpu_rt) { + read_unlock_bh(&table->tb6_lock); + } else { +- /* atomic_inc_not_zero() is needed when using rcu */ +- if (atomic_inc_not_zero(&rt->rt6i_ref)) { +- /* We have to do the read_unlock first +- * because rt6_make_pcpu_route() may trigger +- * ip6_dst_gc() which will take the write_lock. +- * +- * No dst_hold() on rt is needed because grabbing +- * rt->rt6i_ref makes sure rt can't be released. +- */ +- read_unlock_bh(&table->tb6_lock); +- pcpu_rt = rt6_make_pcpu_route(rt); +- rt6_release(rt); +- } else { +- /* rt is already removed from tree */ +- read_unlock_bh(&table->tb6_lock); +- pcpu_rt = net->ipv6.ip6_null_entry; +- dst_hold(&pcpu_rt->dst); +- } ++ /* We have to do the read_unlock first ++ * because rt6_make_pcpu_route() may trigger ++ * ip6_dst_gc() which will take the write_lock. ++ */ ++ dst_hold(&rt->dst); ++ read_unlock_bh(&table->tb6_lock); ++ pcpu_rt = rt6_make_pcpu_route(rt); ++ dst_release(&rt->dst); + } + + trace_fib6_table_lookup(net, pcpu_rt, table->tb6_id, fl6); + return pcpu_rt; ++ + } + } + EXPORT_SYMBOL_GPL(ip6_pol_route); +diff --git a/sound/core/rawmidi.c b/sound/core/rawmidi.c +index b3b353d72527..f055ca10bbc1 100644 +--- a/sound/core/rawmidi.c ++++ b/sound/core/rawmidi.c +@@ -579,15 +579,14 @@ static int snd_rawmidi_info_user(struct snd_rawmidi_substream *substream, + return 0; + } + +-int snd_rawmidi_info_select(struct snd_card *card, struct snd_rawmidi_info *info) ++static int __snd_rawmidi_info_select(struct snd_card *card, ++ struct snd_rawmidi_info *info) + { + struct snd_rawmidi *rmidi; + struct snd_rawmidi_str *pstr; + struct snd_rawmidi_substream *substream; + +- mutex_lock(®ister_mutex); + rmidi = snd_rawmidi_search(card, info->device); +- mutex_unlock(®ister_mutex); + if (!rmidi) + return -ENXIO; + if (info->stream < 0 || info->stream > 1) +@@ -603,6 +602,16 @@ int snd_rawmidi_info_select(struct snd_card *card, struct snd_rawmidi_info *info + } + return -ENXIO; + } ++ ++int snd_rawmidi_info_select(struct snd_card *card, struct snd_rawmidi_info *info) ++{ ++ int ret; ++ ++ mutex_lock(®ister_mutex); ++ ret = __snd_rawmidi_info_select(card, info); ++ mutex_unlock(®ister_mutex); ++ return ret; ++} + EXPORT_SYMBOL(snd_rawmidi_info_select); + + static int snd_rawmidi_info_select_user(struct snd_card *card, +diff --git a/sound/pci/hda/patch_hdmi.c b/sound/pci/hda/patch_hdmi.c +index c19c81d230bd..b4f1b6e88305 100644 +--- a/sound/pci/hda/patch_hdmi.c ++++ b/sound/pci/hda/patch_hdmi.c +@@ -55,10 +55,11 @@ MODULE_PARM_DESC(static_hdmi_pcm, "Don't restrict PCM parameters per ELD info"); + #define is_kabylake(codec) ((codec)->core.vendor_id == 0x8086280b) + #define is_geminilake(codec) (((codec)->core.vendor_id == 0x8086280d) || \ + ((codec)->core.vendor_id == 0x80862800)) ++#define is_cannonlake(codec) ((codec)->core.vendor_id == 0x8086280c) + #define is_haswell_plus(codec) (is_haswell(codec) || is_broadwell(codec) \ + || is_skylake(codec) || is_broxton(codec) \ +- || is_kabylake(codec)) || is_geminilake(codec) +- ++ || is_kabylake(codec)) || is_geminilake(codec) \ ++ || is_cannonlake(codec) + #define is_valleyview(codec) ((codec)->core.vendor_id == 0x80862882) + #define is_cherryview(codec) ((codec)->core.vendor_id == 0x80862883) + #define is_valleyview_plus(codec) (is_valleyview(codec) || is_cherryview(codec)) +@@ -3841,6 +3842,7 @@ HDA_CODEC_ENTRY(0x80862808, "Broadwell HDMI", patch_i915_hsw_hdmi), + HDA_CODEC_ENTRY(0x80862809, "Skylake HDMI", patch_i915_hsw_hdmi), + HDA_CODEC_ENTRY(0x8086280a, "Broxton HDMI", patch_i915_hsw_hdmi), + HDA_CODEC_ENTRY(0x8086280b, "Kabylake HDMI", patch_i915_hsw_hdmi), ++HDA_CODEC_ENTRY(0x8086280c, "Cannonlake HDMI", patch_i915_glk_hdmi), + HDA_CODEC_ENTRY(0x8086280d, "Geminilake HDMI", patch_i915_glk_hdmi), + HDA_CODEC_ENTRY(0x80862800, "Geminilake HDMI", patch_i915_glk_hdmi), + HDA_CODEC_ENTRY(0x80862880, "CedarTrail HDMI", patch_generic_hdmi), +diff --git a/sound/pci/hda/patch_realtek.c b/sound/pci/hda/patch_realtek.c +index b076386c8952..9ac4b9076ee2 100644 +--- a/sound/pci/hda/patch_realtek.c ++++ b/sound/pci/hda/patch_realtek.c +@@ -5162,6 +5162,22 @@ static void alc233_alc662_fixup_lenovo_dual_codecs(struct hda_codec *codec, + } + } + ++/* Forcibly assign NID 0x03 to HP/LO while NID 0x02 to SPK for EQ */ ++static void alc274_fixup_bind_dacs(struct hda_codec *codec, ++ const struct hda_fixup *fix, int action) ++{ ++ struct alc_spec *spec = codec->spec; ++ static hda_nid_t preferred_pairs[] = { ++ 0x21, 0x03, 0x1b, 0x03, 0x16, 0x02, ++ 0 ++ }; ++ ++ if (action != HDA_FIXUP_ACT_PRE_PROBE) ++ return; ++ ++ spec->gen.preferred_dacs = preferred_pairs; ++} ++ + /* for hda_fixup_thinkpad_acpi() */ + #include "thinkpad_helper.c" + +@@ -5279,6 +5295,8 @@ enum { + ALC233_FIXUP_LENOVO_MULTI_CODECS, + ALC294_FIXUP_LENOVO_MIC_LOCATION, + ALC700_FIXUP_INTEL_REFERENCE, ++ ALC274_FIXUP_DELL_BIND_DACS, ++ ALC274_FIXUP_DELL_AIO_LINEOUT_VERB, + }; + + static const struct hda_fixup alc269_fixups[] = { +@@ -6089,6 +6107,21 @@ static const struct hda_fixup alc269_fixups[] = { + {} + } + }, ++ [ALC274_FIXUP_DELL_BIND_DACS] = { ++ .type = HDA_FIXUP_FUNC, ++ .v.func = alc274_fixup_bind_dacs, ++ .chained = true, ++ .chain_id = ALC269_FIXUP_DELL1_MIC_NO_PRESENCE ++ }, ++ [ALC274_FIXUP_DELL_AIO_LINEOUT_VERB] = { ++ .type = HDA_FIXUP_PINS, ++ .v.pins = (const struct hda_pintbl[]) { ++ { 0x1b, 0x0401102f }, ++ { } ++ }, ++ .chained = true, ++ .chain_id = ALC274_FIXUP_DELL_BIND_DACS ++ }, + }; + + static const struct snd_pci_quirk alc269_fixup_tbl[] = { +@@ -6550,7 +6583,7 @@ static const struct snd_hda_pin_quirk alc269_pin_fixup_tbl[] = { + {0x14, 0x90170110}, + {0x1b, 0x90a70130}, + {0x21, 0x03211020}), +- SND_HDA_PIN_QUIRK(0x10ec0274, 0x1028, "Dell", ALC269_FIXUP_DELL1_MIC_NO_PRESENCE, ++ SND_HDA_PIN_QUIRK(0x10ec0274, 0x1028, "Dell", ALC274_FIXUP_DELL_AIO_LINEOUT_VERB, + {0x12, 0xb7a60130}, + {0x13, 0xb8a61140}, + {0x16, 0x90170110}, +diff --git a/sound/usb/mixer.c b/sound/usb/mixer.c +index 4fde4f8d4444..75bce127d768 100644 +--- a/sound/usb/mixer.c ++++ b/sound/usb/mixer.c +@@ -2173,20 +2173,25 @@ static int parse_audio_selector_unit(struct mixer_build *state, int unitid, + kctl->private_value = (unsigned long)namelist; + kctl->private_free = usb_mixer_selector_elem_free; + +- nameid = uac_selector_unit_iSelector(desc); ++ /* check the static mapping table at first */ + len = check_mapped_name(map, kctl->id.name, sizeof(kctl->id.name)); +- if (len) +- ; +- else if (nameid) +- len = snd_usb_copy_string_desc(state, nameid, kctl->id.name, +- sizeof(kctl->id.name)); +- else +- len = get_term_name(state, &state->oterm, +- kctl->id.name, sizeof(kctl->id.name), 0); +- + if (!len) { +- strlcpy(kctl->id.name, "USB", sizeof(kctl->id.name)); ++ /* no mapping ? */ ++ /* if iSelector is given, use it */ ++ nameid = uac_selector_unit_iSelector(desc); ++ if (nameid) ++ len = snd_usb_copy_string_desc(state, nameid, ++ kctl->id.name, ++ sizeof(kctl->id.name)); ++ /* ... or pick up the terminal name at next */ ++ if (!len) ++ len = get_term_name(state, &state->oterm, ++ kctl->id.name, sizeof(kctl->id.name), 0); ++ /* ... or use the fixed string "USB" as the last resort */ ++ if (!len) ++ strlcpy(kctl->id.name, "USB", sizeof(kctl->id.name)); + ++ /* and add the proper suffix */ + if (desc->bDescriptorSubtype == UAC2_CLOCK_SELECTOR) + append_ctl_name(kctl, " Clock Source"); + else if ((state->oterm.type & 0xff00) == 0x0100) +diff --git a/sound/usb/quirks.c b/sound/usb/quirks.c +index 20624320b753..8d7db7cd4f88 100644 +--- a/sound/usb/quirks.c ++++ b/sound/usb/quirks.c +@@ -1172,10 +1172,11 @@ static bool is_marantz_denon_dac(unsigned int id) + /* TEAC UD-501/UD-503/NT-503 USB DACs need a vendor cmd to switch + * between PCM/DOP and native DSD mode + */ +-static bool is_teac_50X_dac(unsigned int id) ++static bool is_teac_dsd_dac(unsigned int id) + { + switch (id) { + case USB_ID(0x0644, 0x8043): /* TEAC UD-501/UD-503/NT-503 */ ++ case USB_ID(0x0644, 0x8044): /* Esoteric D-05X */ + return true; + } + return false; +@@ -1208,7 +1209,7 @@ int snd_usb_select_mode_quirk(struct snd_usb_substream *subs, + break; + } + mdelay(20); +- } else if (is_teac_50X_dac(subs->stream->chip->usb_id)) { ++ } else if (is_teac_dsd_dac(subs->stream->chip->usb_id)) { + /* Vendor mode switch cmd is required. */ + switch (fmt->altsetting) { + case 3: /* DSD mode (DSD_U32) requested */ +@@ -1398,7 +1399,7 @@ u64 snd_usb_interface_dsd_format_quirks(struct snd_usb_audio *chip, + } + + /* TEAC devices with USB DAC functionality */ +- if (is_teac_50X_dac(chip->usb_id)) { ++ if (is_teac_dsd_dac(chip->usb_id)) { + if (fp->altsetting == 3) + return SNDRV_PCM_FMTBIT_DSD_U32_BE; + } +diff --git a/tools/objtool/.gitignore b/tools/objtool/.gitignore +index d3102c865a95..914cff12899b 100644 +--- a/tools/objtool/.gitignore ++++ b/tools/objtool/.gitignore +@@ -1,3 +1,3 @@ +-arch/x86/insn/inat-tables.c ++arch/x86/lib/inat-tables.c + objtool + fixdep +diff --git a/tools/objtool/Makefile b/tools/objtool/Makefile +index 424b1965d06f..ae0272f9a091 100644 +--- a/tools/objtool/Makefile ++++ b/tools/objtool/Makefile +@@ -7,9 +7,11 @@ ARCH := x86 + endif + + # always use the host compiler +-CC = gcc +-LD = ld +-AR = ar ++HOSTCC ?= gcc ++HOSTLD ?= ld ++CC = $(HOSTCC) ++LD = $(HOSTLD) ++AR = ar + + ifeq ($(srctree),) + srctree := $(patsubst %/,%,$(dir $(CURDIR))) +@@ -25,7 +27,9 @@ OBJTOOL_IN := $(OBJTOOL)-in.o + + all: $(OBJTOOL) + +-INCLUDES := -I$(srctree)/tools/include -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi ++INCLUDES := -I$(srctree)/tools/include \ ++ -I$(srctree)/tools/arch/$(HOSTARCH)/include/uapi \ ++ -I$(srctree)/tools/objtool/arch/$(ARCH)/include + WARNINGS := $(EXTRA_WARNINGS) -Wno-switch-default -Wno-switch-enum -Wno-packed + CFLAGS += -Wall -Werror $(WARNINGS) -fomit-frame-pointer -O2 -g $(INCLUDES) + LDFLAGS += -lelf $(LIBSUBCMD) +@@ -41,22 +45,8 @@ include $(srctree)/tools/build/Makefile.include + $(OBJTOOL_IN): fixdep FORCE + @$(MAKE) $(build)=objtool + +-# Busybox's diff doesn't have -I, avoid warning in that case +-# + $(OBJTOOL): $(LIBSUBCMD) $(OBJTOOL_IN) +- @(diff -I 2>&1 | grep -q 'option requires an argument' && \ +- test -d ../../kernel -a -d ../../tools -a -d ../objtool && (( \ +- diff -I'^#include' arch/x86/insn/insn.c ../../arch/x86/lib/insn.c >/dev/null && \ +- diff -I'^#include' arch/x86/insn/inat.c ../../arch/x86/lib/inat.c >/dev/null && \ +- diff arch/x86/insn/x86-opcode-map.txt ../../arch/x86/lib/x86-opcode-map.txt >/dev/null && \ +- diff arch/x86/insn/gen-insn-attr-x86.awk ../../arch/x86/tools/gen-insn-attr-x86.awk >/dev/null && \ +- diff -I'^#include' arch/x86/insn/insn.h ../../arch/x86/include/asm/insn.h >/dev/null && \ +- diff -I'^#include' arch/x86/insn/inat.h ../../arch/x86/include/asm/inat.h >/dev/null && \ +- diff -I'^#include' arch/x86/insn/inat_types.h ../../arch/x86/include/asm/inat_types.h >/dev/null) \ +- || echo "warning: objtool: x86 instruction decoder differs from kernel" >&2 )) || true +- @(test -d ../../kernel -a -d ../../tools -a -d ../objtool && (( \ +- diff ../../arch/x86/include/asm/orc_types.h orc_types.h >/dev/null) \ +- || echo "warning: objtool: orc_types.h differs from kernel" >&2 )) || true ++ @./sync-check.sh + $(QUIET_LINK)$(CC) $(OBJTOOL_IN) $(LDFLAGS) -o $@ + + +@@ -66,7 +56,7 @@ $(LIBSUBCMD): fixdep FORCE + clean: + $(call QUIET_CLEAN, objtool) $(RM) $(OBJTOOL) + $(Q)find $(OUTPUT) -name '*.o' -delete -o -name '\.*.cmd' -delete -o -name '\.*.d' -delete +- $(Q)$(RM) $(OUTPUT)arch/x86/insn/inat-tables.c $(OUTPUT)fixdep ++ $(Q)$(RM) $(OUTPUT)arch/x86/lib/inat-tables.c $(OUTPUT)fixdep + + FORCE: + +diff --git a/tools/objtool/arch/x86/Build b/tools/objtool/arch/x86/Build +index debbdb0b5c43..b998412c017d 100644 +--- a/tools/objtool/arch/x86/Build ++++ b/tools/objtool/arch/x86/Build +@@ -1,12 +1,12 @@ + objtool-y += decode.o + +-inat_tables_script = arch/x86/insn/gen-insn-attr-x86.awk +-inat_tables_maps = arch/x86/insn/x86-opcode-map.txt ++inat_tables_script = arch/x86/tools/gen-insn-attr-x86.awk ++inat_tables_maps = arch/x86/lib/x86-opcode-map.txt + +-$(OUTPUT)arch/x86/insn/inat-tables.c: $(inat_tables_script) $(inat_tables_maps) ++$(OUTPUT)arch/x86/lib/inat-tables.c: $(inat_tables_script) $(inat_tables_maps) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(AWK) -f $(inat_tables_script) $(inat_tables_maps) > $@ + +-$(OUTPUT)arch/x86/decode.o: $(OUTPUT)arch/x86/insn/inat-tables.c ++$(OUTPUT)arch/x86/decode.o: $(OUTPUT)arch/x86/lib/inat-tables.c + +-CFLAGS_decode.o += -I$(OUTPUT)arch/x86/insn ++CFLAGS_decode.o += -I$(OUTPUT)arch/x86/lib +diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c +index 34a579f806e3..8acfc47af70e 100644 +--- a/tools/objtool/arch/x86/decode.c ++++ b/tools/objtool/arch/x86/decode.c +@@ -19,9 +19,9 @@ + #include + + #define unlikely(cond) (cond) +-#include "insn/insn.h" +-#include "insn/inat.c" +-#include "insn/insn.c" ++#include ++#include "lib/inat.c" ++#include "lib/insn.c" + + #include "../../elf.h" + #include "../../arch.h" +diff --git a/tools/objtool/arch/x86/include/asm/inat.h b/tools/objtool/arch/x86/include/asm/inat.h +new file mode 100644 +index 000000000000..1c78580e58be +--- /dev/null ++++ b/tools/objtool/arch/x86/include/asm/inat.h +@@ -0,0 +1,244 @@ ++#ifndef _ASM_X86_INAT_H ++#define _ASM_X86_INAT_H ++/* ++ * x86 instruction attributes ++ * ++ * Written by Masami Hiramatsu ++ * ++ * This program is free software; you can redistribute it and/or modify ++ * it under the terms of the GNU General Public License as published by ++ * the Free Software Foundation; either version 2 of the License, or ++ * (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, write to the Free Software ++ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ++ * ++ */ ++#include ++ ++/* ++ * Internal bits. Don't use bitmasks directly, because these bits are ++ * unstable. You should use checking functions. ++ */ ++ ++#define INAT_OPCODE_TABLE_SIZE 256 ++#define INAT_GROUP_TABLE_SIZE 8 ++ ++/* Legacy last prefixes */ ++#define INAT_PFX_OPNDSZ 1 /* 0x66 */ /* LPFX1 */ ++#define INAT_PFX_REPE 2 /* 0xF3 */ /* LPFX2 */ ++#define INAT_PFX_REPNE 3 /* 0xF2 */ /* LPFX3 */ ++/* Other Legacy prefixes */ ++#define INAT_PFX_LOCK 4 /* 0xF0 */ ++#define INAT_PFX_CS 5 /* 0x2E */ ++#define INAT_PFX_DS 6 /* 0x3E */ ++#define INAT_PFX_ES 7 /* 0x26 */ ++#define INAT_PFX_FS 8 /* 0x64 */ ++#define INAT_PFX_GS 9 /* 0x65 */ ++#define INAT_PFX_SS 10 /* 0x36 */ ++#define INAT_PFX_ADDRSZ 11 /* 0x67 */ ++/* x86-64 REX prefix */ ++#define INAT_PFX_REX 12 /* 0x4X */ ++/* AVX VEX prefixes */ ++#define INAT_PFX_VEX2 13 /* 2-bytes VEX prefix */ ++#define INAT_PFX_VEX3 14 /* 3-bytes VEX prefix */ ++#define INAT_PFX_EVEX 15 /* EVEX prefix */ ++ ++#define INAT_LSTPFX_MAX 3 ++#define INAT_LGCPFX_MAX 11 ++ ++/* Immediate size */ ++#define INAT_IMM_BYTE 1 ++#define INAT_IMM_WORD 2 ++#define INAT_IMM_DWORD 3 ++#define INAT_IMM_QWORD 4 ++#define INAT_IMM_PTR 5 ++#define INAT_IMM_VWORD32 6 ++#define INAT_IMM_VWORD 7 ++ ++/* Legacy prefix */ ++#define INAT_PFX_OFFS 0 ++#define INAT_PFX_BITS 4 ++#define INAT_PFX_MAX ((1 << INAT_PFX_BITS) - 1) ++#define INAT_PFX_MASK (INAT_PFX_MAX << INAT_PFX_OFFS) ++/* Escape opcodes */ ++#define INAT_ESC_OFFS (INAT_PFX_OFFS + INAT_PFX_BITS) ++#define INAT_ESC_BITS 2 ++#define INAT_ESC_MAX ((1 << INAT_ESC_BITS) - 1) ++#define INAT_ESC_MASK (INAT_ESC_MAX << INAT_ESC_OFFS) ++/* Group opcodes (1-16) */ ++#define INAT_GRP_OFFS (INAT_ESC_OFFS + INAT_ESC_BITS) ++#define INAT_GRP_BITS 5 ++#define INAT_GRP_MAX ((1 << INAT_GRP_BITS) - 1) ++#define INAT_GRP_MASK (INAT_GRP_MAX << INAT_GRP_OFFS) ++/* Immediates */ ++#define INAT_IMM_OFFS (INAT_GRP_OFFS + INAT_GRP_BITS) ++#define INAT_IMM_BITS 3 ++#define INAT_IMM_MASK (((1 << INAT_IMM_BITS) - 1) << INAT_IMM_OFFS) ++/* Flags */ ++#define INAT_FLAG_OFFS (INAT_IMM_OFFS + INAT_IMM_BITS) ++#define INAT_MODRM (1 << (INAT_FLAG_OFFS)) ++#define INAT_FORCE64 (1 << (INAT_FLAG_OFFS + 1)) ++#define INAT_SCNDIMM (1 << (INAT_FLAG_OFFS + 2)) ++#define INAT_MOFFSET (1 << (INAT_FLAG_OFFS + 3)) ++#define INAT_VARIANT (1 << (INAT_FLAG_OFFS + 4)) ++#define INAT_VEXOK (1 << (INAT_FLAG_OFFS + 5)) ++#define INAT_VEXONLY (1 << (INAT_FLAG_OFFS + 6)) ++#define INAT_EVEXONLY (1 << (INAT_FLAG_OFFS + 7)) ++/* Attribute making macros for attribute tables */ ++#define INAT_MAKE_PREFIX(pfx) (pfx << INAT_PFX_OFFS) ++#define INAT_MAKE_ESCAPE(esc) (esc << INAT_ESC_OFFS) ++#define INAT_MAKE_GROUP(grp) ((grp << INAT_GRP_OFFS) | INAT_MODRM) ++#define INAT_MAKE_IMM(imm) (imm << INAT_IMM_OFFS) ++ ++/* Identifiers for segment registers */ ++#define INAT_SEG_REG_IGNORE 0 ++#define INAT_SEG_REG_DEFAULT 1 ++#define INAT_SEG_REG_CS 2 ++#define INAT_SEG_REG_SS 3 ++#define INAT_SEG_REG_DS 4 ++#define INAT_SEG_REG_ES 5 ++#define INAT_SEG_REG_FS 6 ++#define INAT_SEG_REG_GS 7 ++ ++/* Attribute search APIs */ ++extern insn_attr_t inat_get_opcode_attribute(insn_byte_t opcode); ++extern int inat_get_last_prefix_id(insn_byte_t last_pfx); ++extern insn_attr_t inat_get_escape_attribute(insn_byte_t opcode, ++ int lpfx_id, ++ insn_attr_t esc_attr); ++extern insn_attr_t inat_get_group_attribute(insn_byte_t modrm, ++ int lpfx_id, ++ insn_attr_t esc_attr); ++extern insn_attr_t inat_get_avx_attribute(insn_byte_t opcode, ++ insn_byte_t vex_m, ++ insn_byte_t vex_pp); ++ ++/* Attribute checking functions */ ++static inline int inat_is_legacy_prefix(insn_attr_t attr) ++{ ++ attr &= INAT_PFX_MASK; ++ return attr && attr <= INAT_LGCPFX_MAX; ++} ++ ++static inline int inat_is_address_size_prefix(insn_attr_t attr) ++{ ++ return (attr & INAT_PFX_MASK) == INAT_PFX_ADDRSZ; ++} ++ ++static inline int inat_is_operand_size_prefix(insn_attr_t attr) ++{ ++ return (attr & INAT_PFX_MASK) == INAT_PFX_OPNDSZ; ++} ++ ++static inline int inat_is_rex_prefix(insn_attr_t attr) ++{ ++ return (attr & INAT_PFX_MASK) == INAT_PFX_REX; ++} ++ ++static inline int inat_last_prefix_id(insn_attr_t attr) ++{ ++ if ((attr & INAT_PFX_MASK) > INAT_LSTPFX_MAX) ++ return 0; ++ else ++ return attr & INAT_PFX_MASK; ++} ++ ++static inline int inat_is_vex_prefix(insn_attr_t attr) ++{ ++ attr &= INAT_PFX_MASK; ++ return attr == INAT_PFX_VEX2 || attr == INAT_PFX_VEX3 || ++ attr == INAT_PFX_EVEX; ++} ++ ++static inline int inat_is_evex_prefix(insn_attr_t attr) ++{ ++ return (attr & INAT_PFX_MASK) == INAT_PFX_EVEX; ++} ++ ++static inline int inat_is_vex3_prefix(insn_attr_t attr) ++{ ++ return (attr & INAT_PFX_MASK) == INAT_PFX_VEX3; ++} ++ ++static inline int inat_is_escape(insn_attr_t attr) ++{ ++ return attr & INAT_ESC_MASK; ++} ++ ++static inline int inat_escape_id(insn_attr_t attr) ++{ ++ return (attr & INAT_ESC_MASK) >> INAT_ESC_OFFS; ++} ++ ++static inline int inat_is_group(insn_attr_t attr) ++{ ++ return attr & INAT_GRP_MASK; ++} ++ ++static inline int inat_group_id(insn_attr_t attr) ++{ ++ return (attr & INAT_GRP_MASK) >> INAT_GRP_OFFS; ++} ++ ++static inline int inat_group_common_attribute(insn_attr_t attr) ++{ ++ return attr & ~INAT_GRP_MASK; ++} ++ ++static inline int inat_has_immediate(insn_attr_t attr) ++{ ++ return attr & INAT_IMM_MASK; ++} ++ ++static inline int inat_immediate_size(insn_attr_t attr) ++{ ++ return (attr & INAT_IMM_MASK) >> INAT_IMM_OFFS; ++} ++ ++static inline int inat_has_modrm(insn_attr_t attr) ++{ ++ return attr & INAT_MODRM; ++} ++ ++static inline int inat_is_force64(insn_attr_t attr) ++{ ++ return attr & INAT_FORCE64; ++} ++ ++static inline int inat_has_second_immediate(insn_attr_t attr) ++{ ++ return attr & INAT_SCNDIMM; ++} ++ ++static inline int inat_has_moffset(insn_attr_t attr) ++{ ++ return attr & INAT_MOFFSET; ++} ++ ++static inline int inat_has_variant(insn_attr_t attr) ++{ ++ return attr & INAT_VARIANT; ++} ++ ++static inline int inat_accept_vex(insn_attr_t attr) ++{ ++ return attr & INAT_VEXOK; ++} ++ ++static inline int inat_must_vex(insn_attr_t attr) ++{ ++ return attr & (INAT_VEXONLY | INAT_EVEXONLY); ++} ++ ++static inline int inat_must_evex(insn_attr_t attr) ++{ ++ return attr & INAT_EVEXONLY; ++} ++#endif +diff --git a/tools/objtool/arch/x86/include/asm/inat_types.h b/tools/objtool/arch/x86/include/asm/inat_types.h +new file mode 100644 +index 000000000000..cb3c20ce39cf +--- /dev/null ++++ b/tools/objtool/arch/x86/include/asm/inat_types.h +@@ -0,0 +1,29 @@ ++#ifndef _ASM_X86_INAT_TYPES_H ++#define _ASM_X86_INAT_TYPES_H ++/* ++ * x86 instruction attributes ++ * ++ * Written by Masami Hiramatsu ++ * ++ * This program is free software; you can redistribute it and/or modify ++ * it under the terms of the GNU General Public License as published by ++ * the Free Software Foundation; either version 2 of the License, or ++ * (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, write to the Free Software ++ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ++ * ++ */ ++ ++/* Instruction attributes */ ++typedef unsigned int insn_attr_t; ++typedef unsigned char insn_byte_t; ++typedef signed int insn_value_t; ++ ++#endif +diff --git a/tools/objtool/arch/x86/include/asm/insn.h b/tools/objtool/arch/x86/include/asm/insn.h +new file mode 100644 +index 000000000000..b3e32b010ab1 +--- /dev/null ++++ b/tools/objtool/arch/x86/include/asm/insn.h +@@ -0,0 +1,211 @@ ++#ifndef _ASM_X86_INSN_H ++#define _ASM_X86_INSN_H ++/* ++ * x86 instruction analysis ++ * ++ * This program is free software; you can redistribute it and/or modify ++ * it under the terms of the GNU General Public License as published by ++ * the Free Software Foundation; either version 2 of the License, or ++ * (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, write to the Free Software ++ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ++ * ++ * Copyright (C) IBM Corporation, 2009 ++ */ ++ ++/* insn_attr_t is defined in inat.h */ ++#include ++ ++struct insn_field { ++ union { ++ insn_value_t value; ++ insn_byte_t bytes[4]; ++ }; ++ /* !0 if we've run insn_get_xxx() for this field */ ++ unsigned char got; ++ unsigned char nbytes; ++}; ++ ++struct insn { ++ struct insn_field prefixes; /* ++ * Prefixes ++ * prefixes.bytes[3]: last prefix ++ */ ++ struct insn_field rex_prefix; /* REX prefix */ ++ struct insn_field vex_prefix; /* VEX prefix */ ++ struct insn_field opcode; /* ++ * opcode.bytes[0]: opcode1 ++ * opcode.bytes[1]: opcode2 ++ * opcode.bytes[2]: opcode3 ++ */ ++ struct insn_field modrm; ++ struct insn_field sib; ++ struct insn_field displacement; ++ union { ++ struct insn_field immediate; ++ struct insn_field moffset1; /* for 64bit MOV */ ++ struct insn_field immediate1; /* for 64bit imm or off16/32 */ ++ }; ++ union { ++ struct insn_field moffset2; /* for 64bit MOV */ ++ struct insn_field immediate2; /* for 64bit imm or seg16 */ ++ }; ++ ++ insn_attr_t attr; ++ unsigned char opnd_bytes; ++ unsigned char addr_bytes; ++ unsigned char length; ++ unsigned char x86_64; ++ ++ const insn_byte_t *kaddr; /* kernel address of insn to analyze */ ++ const insn_byte_t *end_kaddr; /* kernel address of last insn in buffer */ ++ const insn_byte_t *next_byte; ++}; ++ ++#define MAX_INSN_SIZE 15 ++ ++#define X86_MODRM_MOD(modrm) (((modrm) & 0xc0) >> 6) ++#define X86_MODRM_REG(modrm) (((modrm) & 0x38) >> 3) ++#define X86_MODRM_RM(modrm) ((modrm) & 0x07) ++ ++#define X86_SIB_SCALE(sib) (((sib) & 0xc0) >> 6) ++#define X86_SIB_INDEX(sib) (((sib) & 0x38) >> 3) ++#define X86_SIB_BASE(sib) ((sib) & 0x07) ++ ++#define X86_REX_W(rex) ((rex) & 8) ++#define X86_REX_R(rex) ((rex) & 4) ++#define X86_REX_X(rex) ((rex) & 2) ++#define X86_REX_B(rex) ((rex) & 1) ++ ++/* VEX bit flags */ ++#define X86_VEX_W(vex) ((vex) & 0x80) /* VEX3 Byte2 */ ++#define X86_VEX_R(vex) ((vex) & 0x80) /* VEX2/3 Byte1 */ ++#define X86_VEX_X(vex) ((vex) & 0x40) /* VEX3 Byte1 */ ++#define X86_VEX_B(vex) ((vex) & 0x20) /* VEX3 Byte1 */ ++#define X86_VEX_L(vex) ((vex) & 0x04) /* VEX3 Byte2, VEX2 Byte1 */ ++/* VEX bit fields */ ++#define X86_EVEX_M(vex) ((vex) & 0x03) /* EVEX Byte1 */ ++#define X86_VEX3_M(vex) ((vex) & 0x1f) /* VEX3 Byte1 */ ++#define X86_VEX2_M 1 /* VEX2.M always 1 */ ++#define X86_VEX_V(vex) (((vex) & 0x78) >> 3) /* VEX3 Byte2, VEX2 Byte1 */ ++#define X86_VEX_P(vex) ((vex) & 0x03) /* VEX3 Byte2, VEX2 Byte1 */ ++#define X86_VEX_M_MAX 0x1f /* VEX3.M Maximum value */ ++ ++extern void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64); ++extern void insn_get_prefixes(struct insn *insn); ++extern void insn_get_opcode(struct insn *insn); ++extern void insn_get_modrm(struct insn *insn); ++extern void insn_get_sib(struct insn *insn); ++extern void insn_get_displacement(struct insn *insn); ++extern void insn_get_immediate(struct insn *insn); ++extern void insn_get_length(struct insn *insn); ++ ++/* Attribute will be determined after getting ModRM (for opcode groups) */ ++static inline void insn_get_attribute(struct insn *insn) ++{ ++ insn_get_modrm(insn); ++} ++ ++/* Instruction uses RIP-relative addressing */ ++extern int insn_rip_relative(struct insn *insn); ++ ++/* Init insn for kernel text */ ++static inline void kernel_insn_init(struct insn *insn, ++ const void *kaddr, int buf_len) ++{ ++#ifdef CONFIG_X86_64 ++ insn_init(insn, kaddr, buf_len, 1); ++#else /* CONFIG_X86_32 */ ++ insn_init(insn, kaddr, buf_len, 0); ++#endif ++} ++ ++static inline int insn_is_avx(struct insn *insn) ++{ ++ if (!insn->prefixes.got) ++ insn_get_prefixes(insn); ++ return (insn->vex_prefix.value != 0); ++} ++ ++static inline int insn_is_evex(struct insn *insn) ++{ ++ if (!insn->prefixes.got) ++ insn_get_prefixes(insn); ++ return (insn->vex_prefix.nbytes == 4); ++} ++ ++/* Ensure this instruction is decoded completely */ ++static inline int insn_complete(struct insn *insn) ++{ ++ return insn->opcode.got && insn->modrm.got && insn->sib.got && ++ insn->displacement.got && insn->immediate.got; ++} ++ ++static inline insn_byte_t insn_vex_m_bits(struct insn *insn) ++{ ++ if (insn->vex_prefix.nbytes == 2) /* 2 bytes VEX */ ++ return X86_VEX2_M; ++ else if (insn->vex_prefix.nbytes == 3) /* 3 bytes VEX */ ++ return X86_VEX3_M(insn->vex_prefix.bytes[1]); ++ else /* EVEX */ ++ return X86_EVEX_M(insn->vex_prefix.bytes[1]); ++} ++ ++static inline insn_byte_t insn_vex_p_bits(struct insn *insn) ++{ ++ if (insn->vex_prefix.nbytes == 2) /* 2 bytes VEX */ ++ return X86_VEX_P(insn->vex_prefix.bytes[1]); ++ else ++ return X86_VEX_P(insn->vex_prefix.bytes[2]); ++} ++ ++/* Get the last prefix id from last prefix or VEX prefix */ ++static inline int insn_last_prefix_id(struct insn *insn) ++{ ++ if (insn_is_avx(insn)) ++ return insn_vex_p_bits(insn); /* VEX_p is a SIMD prefix id */ ++ ++ if (insn->prefixes.bytes[3]) ++ return inat_get_last_prefix_id(insn->prefixes.bytes[3]); ++ ++ return 0; ++} ++ ++/* Offset of each field from kaddr */ ++static inline int insn_offset_rex_prefix(struct insn *insn) ++{ ++ return insn->prefixes.nbytes; ++} ++static inline int insn_offset_vex_prefix(struct insn *insn) ++{ ++ return insn_offset_rex_prefix(insn) + insn->rex_prefix.nbytes; ++} ++static inline int insn_offset_opcode(struct insn *insn) ++{ ++ return insn_offset_vex_prefix(insn) + insn->vex_prefix.nbytes; ++} ++static inline int insn_offset_modrm(struct insn *insn) ++{ ++ return insn_offset_opcode(insn) + insn->opcode.nbytes; ++} ++static inline int insn_offset_sib(struct insn *insn) ++{ ++ return insn_offset_modrm(insn) + insn->modrm.nbytes; ++} ++static inline int insn_offset_displacement(struct insn *insn) ++{ ++ return insn_offset_sib(insn) + insn->sib.nbytes; ++} ++static inline int insn_offset_immediate(struct insn *insn) ++{ ++ return insn_offset_displacement(insn) + insn->displacement.nbytes; ++} ++ ++#endif /* _ASM_X86_INSN_H */ +diff --git a/tools/objtool/arch/x86/include/asm/orc_types.h b/tools/objtool/arch/x86/include/asm/orc_types.h +new file mode 100644 +index 000000000000..9c9dc579bd7d +--- /dev/null ++++ b/tools/objtool/arch/x86/include/asm/orc_types.h +@@ -0,0 +1,107 @@ ++/* ++ * Copyright (C) 2017 Josh Poimboeuf ++ * ++ * This program is free software; you can redistribute it and/or ++ * modify it under the terms of the GNU General Public License ++ * as published by the Free Software Foundation; either version 2 ++ * of the License, or (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, see . ++ */ ++ ++#ifndef _ORC_TYPES_H ++#define _ORC_TYPES_H ++ ++#include ++#include ++ ++/* ++ * The ORC_REG_* registers are base registers which are used to find other ++ * registers on the stack. ++ * ++ * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the ++ * address of the previous frame: the caller's SP before it called the current ++ * function. ++ * ++ * ORC_REG_UNDEFINED means the corresponding register's value didn't change in ++ * the current frame. ++ * ++ * The most commonly used base registers are SP and BP -- which the previous SP ++ * is usually based on -- and PREV_SP and UNDEFINED -- which the previous BP is ++ * usually based on. ++ * ++ * The rest of the base registers are needed for special cases like entry code ++ * and GCC realigned stacks. ++ */ ++#define ORC_REG_UNDEFINED 0 ++#define ORC_REG_PREV_SP 1 ++#define ORC_REG_DX 2 ++#define ORC_REG_DI 3 ++#define ORC_REG_BP 4 ++#define ORC_REG_SP 5 ++#define ORC_REG_R10 6 ++#define ORC_REG_R13 7 ++#define ORC_REG_BP_INDIRECT 8 ++#define ORC_REG_SP_INDIRECT 9 ++#define ORC_REG_MAX 15 ++ ++/* ++ * ORC_TYPE_CALL: Indicates that sp_reg+sp_offset resolves to PREV_SP (the ++ * caller's SP right before it made the call). Used for all callable ++ * functions, i.e. all C code and all callable asm functions. ++ * ++ * ORC_TYPE_REGS: Used in entry code to indicate that sp_reg+sp_offset points ++ * to a fully populated pt_regs from a syscall, interrupt, or exception. ++ * ++ * ORC_TYPE_REGS_IRET: Used in entry code to indicate that sp_reg+sp_offset ++ * points to the iret return frame. ++ * ++ * The UNWIND_HINT macros are used only for the unwind_hint struct. They ++ * aren't used in struct orc_entry due to size and complexity constraints. ++ * Objtool converts them to real types when it converts the hints to orc ++ * entries. ++ */ ++#define ORC_TYPE_CALL 0 ++#define ORC_TYPE_REGS 1 ++#define ORC_TYPE_REGS_IRET 2 ++#define UNWIND_HINT_TYPE_SAVE 3 ++#define UNWIND_HINT_TYPE_RESTORE 4 ++ ++#ifndef __ASSEMBLY__ ++/* ++ * This struct is more or less a vastly simplified version of the DWARF Call ++ * Frame Information standard. It contains only the necessary parts of DWARF ++ * CFI, simplified for ease of access by the in-kernel unwinder. It tells the ++ * unwinder how to find the previous SP and BP (and sometimes entry regs) on ++ * the stack for a given code address. Each instance of the struct corresponds ++ * to one or more code locations. ++ */ ++struct orc_entry { ++ s16 sp_offset; ++ s16 bp_offset; ++ unsigned sp_reg:4; ++ unsigned bp_reg:4; ++ unsigned type:2; ++} __packed; ++ ++/* ++ * This struct is used by asm and inline asm code to manually annotate the ++ * location of registers on the stack for the ORC unwinder. ++ * ++ * Type can be either ORC_TYPE_* or UNWIND_HINT_TYPE_*. ++ */ ++struct unwind_hint { ++ u32 ip; ++ s16 sp_offset; ++ u8 sp_reg; ++ u8 type; ++}; ++#endif /* __ASSEMBLY__ */ ++ ++#endif /* _ORC_TYPES_H */ +diff --git a/tools/objtool/arch/x86/insn/gen-insn-attr-x86.awk b/tools/objtool/arch/x86/insn/gen-insn-attr-x86.awk +deleted file mode 100644 +index b02a36b2c14f..000000000000 +--- a/tools/objtool/arch/x86/insn/gen-insn-attr-x86.awk ++++ /dev/null +@@ -1,393 +0,0 @@ +-#!/bin/awk -f +-# SPDX-License-Identifier: GPL-2.0 +-# gen-insn-attr-x86.awk: Instruction attribute table generator +-# Written by Masami Hiramatsu +-# +-# Usage: awk -f gen-insn-attr-x86.awk x86-opcode-map.txt > inat-tables.c +- +-# Awk implementation sanity check +-function check_awk_implement() { +- if (sprintf("%x", 0) != "0") +- return "Your awk has a printf-format problem." +- return "" +-} +- +-# Clear working vars +-function clear_vars() { +- delete table +- delete lptable2 +- delete lptable1 +- delete lptable3 +- eid = -1 # escape id +- gid = -1 # group id +- aid = -1 # AVX id +- tname = "" +-} +- +-BEGIN { +- # Implementation error checking +- awkchecked = check_awk_implement() +- if (awkchecked != "") { +- print "Error: " awkchecked > "/dev/stderr" +- print "Please try to use gawk." > "/dev/stderr" +- exit 1 +- } +- +- # Setup generating tables +- print "/* x86 opcode map generated from x86-opcode-map.txt */" +- print "/* Do not change this code. */\n" +- ggid = 1 +- geid = 1 +- gaid = 0 +- delete etable +- delete gtable +- delete atable +- +- opnd_expr = "^[A-Za-z/]" +- ext_expr = "^\\(" +- sep_expr = "^\\|$" +- group_expr = "^Grp[0-9A-Za-z]+" +- +- imm_expr = "^[IJAOL][a-z]" +- imm_flag["Ib"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" +- imm_flag["Jb"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" +- imm_flag["Iw"] = "INAT_MAKE_IMM(INAT_IMM_WORD)" +- imm_flag["Id"] = "INAT_MAKE_IMM(INAT_IMM_DWORD)" +- imm_flag["Iq"] = "INAT_MAKE_IMM(INAT_IMM_QWORD)" +- imm_flag["Ap"] = "INAT_MAKE_IMM(INAT_IMM_PTR)" +- imm_flag["Iz"] = "INAT_MAKE_IMM(INAT_IMM_VWORD32)" +- imm_flag["Jz"] = "INAT_MAKE_IMM(INAT_IMM_VWORD32)" +- imm_flag["Iv"] = "INAT_MAKE_IMM(INAT_IMM_VWORD)" +- imm_flag["Ob"] = "INAT_MOFFSET" +- imm_flag["Ov"] = "INAT_MOFFSET" +- imm_flag["Lx"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" +- +- modrm_expr = "^([CDEGMNPQRSUVW/][a-z]+|NTA|T[012])" +- force64_expr = "\\([df]64\\)" +- rex_expr = "^REX(\\.[XRWB]+)*" +- fpu_expr = "^ESC" # TODO +- +- lprefix1_expr = "\\((66|!F3)\\)" +- lprefix2_expr = "\\(F3\\)" +- lprefix3_expr = "\\((F2|!F3|66\\&F2)\\)" +- lprefix_expr = "\\((66|F2|F3)\\)" +- max_lprefix = 4 +- +- # All opcodes starting with lower-case 'v', 'k' or with (v1) superscript +- # accepts VEX prefix +- vexok_opcode_expr = "^[vk].*" +- vexok_expr = "\\(v1\\)" +- # All opcodes with (v) superscript supports *only* VEX prefix +- vexonly_expr = "\\(v\\)" +- # All opcodes with (ev) superscript supports *only* EVEX prefix +- evexonly_expr = "\\(ev\\)" +- +- prefix_expr = "\\(Prefix\\)" +- prefix_num["Operand-Size"] = "INAT_PFX_OPNDSZ" +- prefix_num["REPNE"] = "INAT_PFX_REPNE" +- prefix_num["REP/REPE"] = "INAT_PFX_REPE" +- prefix_num["XACQUIRE"] = "INAT_PFX_REPNE" +- prefix_num["XRELEASE"] = "INAT_PFX_REPE" +- prefix_num["LOCK"] = "INAT_PFX_LOCK" +- prefix_num["SEG=CS"] = "INAT_PFX_CS" +- prefix_num["SEG=DS"] = "INAT_PFX_DS" +- prefix_num["SEG=ES"] = "INAT_PFX_ES" +- prefix_num["SEG=FS"] = "INAT_PFX_FS" +- prefix_num["SEG=GS"] = "INAT_PFX_GS" +- prefix_num["SEG=SS"] = "INAT_PFX_SS" +- prefix_num["Address-Size"] = "INAT_PFX_ADDRSZ" +- prefix_num["VEX+1byte"] = "INAT_PFX_VEX2" +- prefix_num["VEX+2byte"] = "INAT_PFX_VEX3" +- prefix_num["EVEX"] = "INAT_PFX_EVEX" +- +- clear_vars() +-} +- +-function semantic_error(msg) { +- print "Semantic error at " NR ": " msg > "/dev/stderr" +- exit 1 +-} +- +-function debug(msg) { +- print "DEBUG: " msg +-} +- +-function array_size(arr, i,c) { +- c = 0 +- for (i in arr) +- c++ +- return c +-} +- +-/^Table:/ { +- print "/* " $0 " */" +- if (tname != "") +- semantic_error("Hit Table: before EndTable:."); +-} +- +-/^Referrer:/ { +- if (NF != 1) { +- # escape opcode table +- ref = "" +- for (i = 2; i <= NF; i++) +- ref = ref $i +- eid = escape[ref] +- tname = sprintf("inat_escape_table_%d", eid) +- } +-} +- +-/^AVXcode:/ { +- if (NF != 1) { +- # AVX/escape opcode table +- aid = $2 +- if (gaid <= aid) +- gaid = aid + 1 +- if (tname == "") # AVX only opcode table +- tname = sprintf("inat_avx_table_%d", $2) +- } +- if (aid == -1 && eid == -1) # primary opcode table +- tname = "inat_primary_table" +-} +- +-/^GrpTable:/ { +- print "/* " $0 " */" +- if (!($2 in group)) +- semantic_error("No group: " $2 ) +- gid = group[$2] +- tname = "inat_group_table_" gid +-} +- +-function print_table(tbl,name,fmt,n) +-{ +- print "const insn_attr_t " name " = {" +- for (i = 0; i < n; i++) { +- id = sprintf(fmt, i) +- if (tbl[id]) +- print " [" id "] = " tbl[id] "," +- } +- print "};" +-} +- +-/^EndTable/ { +- if (gid != -1) { +- # print group tables +- if (array_size(table) != 0) { +- print_table(table, tname "[INAT_GROUP_TABLE_SIZE]", +- "0x%x", 8) +- gtable[gid,0] = tname +- } +- if (array_size(lptable1) != 0) { +- print_table(lptable1, tname "_1[INAT_GROUP_TABLE_SIZE]", +- "0x%x", 8) +- gtable[gid,1] = tname "_1" +- } +- if (array_size(lptable2) != 0) { +- print_table(lptable2, tname "_2[INAT_GROUP_TABLE_SIZE]", +- "0x%x", 8) +- gtable[gid,2] = tname "_2" +- } +- if (array_size(lptable3) != 0) { +- print_table(lptable3, tname "_3[INAT_GROUP_TABLE_SIZE]", +- "0x%x", 8) +- gtable[gid,3] = tname "_3" +- } +- } else { +- # print primary/escaped tables +- if (array_size(table) != 0) { +- print_table(table, tname "[INAT_OPCODE_TABLE_SIZE]", +- "0x%02x", 256) +- etable[eid,0] = tname +- if (aid >= 0) +- atable[aid,0] = tname +- } +- if (array_size(lptable1) != 0) { +- print_table(lptable1,tname "_1[INAT_OPCODE_TABLE_SIZE]", +- "0x%02x", 256) +- etable[eid,1] = tname "_1" +- if (aid >= 0) +- atable[aid,1] = tname "_1" +- } +- if (array_size(lptable2) != 0) { +- print_table(lptable2,tname "_2[INAT_OPCODE_TABLE_SIZE]", +- "0x%02x", 256) +- etable[eid,2] = tname "_2" +- if (aid >= 0) +- atable[aid,2] = tname "_2" +- } +- if (array_size(lptable3) != 0) { +- print_table(lptable3,tname "_3[INAT_OPCODE_TABLE_SIZE]", +- "0x%02x", 256) +- etable[eid,3] = tname "_3" +- if (aid >= 0) +- atable[aid,3] = tname "_3" +- } +- } +- print "" +- clear_vars() +-} +- +-function add_flags(old,new) { +- if (old && new) +- return old " | " new +- else if (old) +- return old +- else +- return new +-} +- +-# convert operands to flags. +-function convert_operands(count,opnd, i,j,imm,mod) +-{ +- imm = null +- mod = null +- for (j = 1; j <= count; j++) { +- i = opnd[j] +- if (match(i, imm_expr) == 1) { +- if (!imm_flag[i]) +- semantic_error("Unknown imm opnd: " i) +- if (imm) { +- if (i != "Ib") +- semantic_error("Second IMM error") +- imm = add_flags(imm, "INAT_SCNDIMM") +- } else +- imm = imm_flag[i] +- } else if (match(i, modrm_expr)) +- mod = "INAT_MODRM" +- } +- return add_flags(imm, mod) +-} +- +-/^[0-9a-f]+\:/ { +- if (NR == 1) +- next +- # get index +- idx = "0x" substr($1, 1, index($1,":") - 1) +- if (idx in table) +- semantic_error("Redefine " idx " in " tname) +- +- # check if escaped opcode +- if ("escape" == $2) { +- if ($3 != "#") +- semantic_error("No escaped name") +- ref = "" +- for (i = 4; i <= NF; i++) +- ref = ref $i +- if (ref in escape) +- semantic_error("Redefine escape (" ref ")") +- escape[ref] = geid +- geid++ +- table[idx] = "INAT_MAKE_ESCAPE(" escape[ref] ")" +- next +- } +- +- variant = null +- # converts +- i = 2 +- while (i <= NF) { +- opcode = $(i++) +- delete opnds +- ext = null +- flags = null +- opnd = null +- # parse one opcode +- if (match($i, opnd_expr)) { +- opnd = $i +- count = split($(i++), opnds, ",") +- flags = convert_operands(count, opnds) +- } +- if (match($i, ext_expr)) +- ext = $(i++) +- if (match($i, sep_expr)) +- i++ +- else if (i < NF) +- semantic_error($i " is not a separator") +- +- # check if group opcode +- if (match(opcode, group_expr)) { +- if (!(opcode in group)) { +- group[opcode] = ggid +- ggid++ +- } +- flags = add_flags(flags, "INAT_MAKE_GROUP(" group[opcode] ")") +- } +- # check force(or default) 64bit +- if (match(ext, force64_expr)) +- flags = add_flags(flags, "INAT_FORCE64") +- +- # check REX prefix +- if (match(opcode, rex_expr)) +- flags = add_flags(flags, "INAT_MAKE_PREFIX(INAT_PFX_REX)") +- +- # check coprocessor escape : TODO +- if (match(opcode, fpu_expr)) +- flags = add_flags(flags, "INAT_MODRM") +- +- # check VEX codes +- if (match(ext, evexonly_expr)) +- flags = add_flags(flags, "INAT_VEXOK | INAT_EVEXONLY") +- else if (match(ext, vexonly_expr)) +- flags = add_flags(flags, "INAT_VEXOK | INAT_VEXONLY") +- else if (match(ext, vexok_expr) || match(opcode, vexok_opcode_expr)) +- flags = add_flags(flags, "INAT_VEXOK") +- +- # check prefixes +- if (match(ext, prefix_expr)) { +- if (!prefix_num[opcode]) +- semantic_error("Unknown prefix: " opcode) +- flags = add_flags(flags, "INAT_MAKE_PREFIX(" prefix_num[opcode] ")") +- } +- if (length(flags) == 0) +- continue +- # check if last prefix +- if (match(ext, lprefix1_expr)) { +- lptable1[idx] = add_flags(lptable1[idx],flags) +- variant = "INAT_VARIANT" +- } +- if (match(ext, lprefix2_expr)) { +- lptable2[idx] = add_flags(lptable2[idx],flags) +- variant = "INAT_VARIANT" +- } +- if (match(ext, lprefix3_expr)) { +- lptable3[idx] = add_flags(lptable3[idx],flags) +- variant = "INAT_VARIANT" +- } +- if (!match(ext, lprefix_expr)){ +- table[idx] = add_flags(table[idx],flags) +- } +- } +- if (variant) +- table[idx] = add_flags(table[idx],variant) +-} +- +-END { +- if (awkchecked != "") +- exit 1 +- # print escape opcode map's array +- print "/* Escape opcode map array */" +- print "const insn_attr_t * const inat_escape_tables[INAT_ESC_MAX + 1]" \ +- "[INAT_LSTPFX_MAX + 1] = {" +- for (i = 0; i < geid; i++) +- for (j = 0; j < max_lprefix; j++) +- if (etable[i,j]) +- print " ["i"]["j"] = "etable[i,j]"," +- print "};\n" +- # print group opcode map's array +- print "/* Group opcode map array */" +- print "const insn_attr_t * const inat_group_tables[INAT_GRP_MAX + 1]"\ +- "[INAT_LSTPFX_MAX + 1] = {" +- for (i = 0; i < ggid; i++) +- for (j = 0; j < max_lprefix; j++) +- if (gtable[i,j]) +- print " ["i"]["j"] = "gtable[i,j]"," +- print "};\n" +- # print AVX opcode map's array +- print "/* AVX opcode map array */" +- print "const insn_attr_t * const inat_avx_tables[X86_VEX_M_MAX + 1]"\ +- "[INAT_LSTPFX_MAX + 1] = {" +- for (i = 0; i < gaid; i++) +- for (j = 0; j < max_lprefix; j++) +- if (atable[i,j]) +- print " ["i"]["j"] = "atable[i,j]"," +- print "};" +-} +- +diff --git a/tools/objtool/arch/x86/insn/inat.c b/tools/objtool/arch/x86/insn/inat.c +deleted file mode 100644 +index e4bf28e6f4c7..000000000000 +--- a/tools/objtool/arch/x86/insn/inat.c ++++ /dev/null +@@ -1,97 +0,0 @@ +-/* +- * x86 instruction attribute tables +- * +- * Written by Masami Hiramatsu +- * +- * This program is free software; you can redistribute it and/or modify +- * it under the terms of the GNU General Public License as published by +- * the Free Software Foundation; either version 2 of the License, or +- * (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, write to the Free Software +- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +- * +- */ +-#include "insn.h" +- +-/* Attribute tables are generated from opcode map */ +-#include "inat-tables.c" +- +-/* Attribute search APIs */ +-insn_attr_t inat_get_opcode_attribute(insn_byte_t opcode) +-{ +- return inat_primary_table[opcode]; +-} +- +-int inat_get_last_prefix_id(insn_byte_t last_pfx) +-{ +- insn_attr_t lpfx_attr; +- +- lpfx_attr = inat_get_opcode_attribute(last_pfx); +- return inat_last_prefix_id(lpfx_attr); +-} +- +-insn_attr_t inat_get_escape_attribute(insn_byte_t opcode, int lpfx_id, +- insn_attr_t esc_attr) +-{ +- const insn_attr_t *table; +- int n; +- +- n = inat_escape_id(esc_attr); +- +- table = inat_escape_tables[n][0]; +- if (!table) +- return 0; +- if (inat_has_variant(table[opcode]) && lpfx_id) { +- table = inat_escape_tables[n][lpfx_id]; +- if (!table) +- return 0; +- } +- return table[opcode]; +-} +- +-insn_attr_t inat_get_group_attribute(insn_byte_t modrm, int lpfx_id, +- insn_attr_t grp_attr) +-{ +- const insn_attr_t *table; +- int n; +- +- n = inat_group_id(grp_attr); +- +- table = inat_group_tables[n][0]; +- if (!table) +- return inat_group_common_attribute(grp_attr); +- if (inat_has_variant(table[X86_MODRM_REG(modrm)]) && lpfx_id) { +- table = inat_group_tables[n][lpfx_id]; +- if (!table) +- return inat_group_common_attribute(grp_attr); +- } +- return table[X86_MODRM_REG(modrm)] | +- inat_group_common_attribute(grp_attr); +-} +- +-insn_attr_t inat_get_avx_attribute(insn_byte_t opcode, insn_byte_t vex_m, +- insn_byte_t vex_p) +-{ +- const insn_attr_t *table; +- if (vex_m > X86_VEX_M_MAX || vex_p > INAT_LSTPFX_MAX) +- return 0; +- /* At first, this checks the master table */ +- table = inat_avx_tables[vex_m][0]; +- if (!table) +- return 0; +- if (!inat_is_group(table[opcode]) && vex_p) { +- /* If this is not a group, get attribute directly */ +- table = inat_avx_tables[vex_m][vex_p]; +- if (!table) +- return 0; +- } +- return table[opcode]; +-} +- +diff --git a/tools/objtool/arch/x86/insn/inat.h b/tools/objtool/arch/x86/insn/inat.h +deleted file mode 100644 +index 125ecd2a300d..000000000000 +--- a/tools/objtool/arch/x86/insn/inat.h ++++ /dev/null +@@ -1,234 +0,0 @@ +-#ifndef _ASM_X86_INAT_H +-#define _ASM_X86_INAT_H +-/* +- * x86 instruction attributes +- * +- * Written by Masami Hiramatsu +- * +- * This program is free software; you can redistribute it and/or modify +- * it under the terms of the GNU General Public License as published by +- * the Free Software Foundation; either version 2 of the License, or +- * (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, write to the Free Software +- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +- * +- */ +-#include "inat_types.h" +- +-/* +- * Internal bits. Don't use bitmasks directly, because these bits are +- * unstable. You should use checking functions. +- */ +- +-#define INAT_OPCODE_TABLE_SIZE 256 +-#define INAT_GROUP_TABLE_SIZE 8 +- +-/* Legacy last prefixes */ +-#define INAT_PFX_OPNDSZ 1 /* 0x66 */ /* LPFX1 */ +-#define INAT_PFX_REPE 2 /* 0xF3 */ /* LPFX2 */ +-#define INAT_PFX_REPNE 3 /* 0xF2 */ /* LPFX3 */ +-/* Other Legacy prefixes */ +-#define INAT_PFX_LOCK 4 /* 0xF0 */ +-#define INAT_PFX_CS 5 /* 0x2E */ +-#define INAT_PFX_DS 6 /* 0x3E */ +-#define INAT_PFX_ES 7 /* 0x26 */ +-#define INAT_PFX_FS 8 /* 0x64 */ +-#define INAT_PFX_GS 9 /* 0x65 */ +-#define INAT_PFX_SS 10 /* 0x36 */ +-#define INAT_PFX_ADDRSZ 11 /* 0x67 */ +-/* x86-64 REX prefix */ +-#define INAT_PFX_REX 12 /* 0x4X */ +-/* AVX VEX prefixes */ +-#define INAT_PFX_VEX2 13 /* 2-bytes VEX prefix */ +-#define INAT_PFX_VEX3 14 /* 3-bytes VEX prefix */ +-#define INAT_PFX_EVEX 15 /* EVEX prefix */ +- +-#define INAT_LSTPFX_MAX 3 +-#define INAT_LGCPFX_MAX 11 +- +-/* Immediate size */ +-#define INAT_IMM_BYTE 1 +-#define INAT_IMM_WORD 2 +-#define INAT_IMM_DWORD 3 +-#define INAT_IMM_QWORD 4 +-#define INAT_IMM_PTR 5 +-#define INAT_IMM_VWORD32 6 +-#define INAT_IMM_VWORD 7 +- +-/* Legacy prefix */ +-#define INAT_PFX_OFFS 0 +-#define INAT_PFX_BITS 4 +-#define INAT_PFX_MAX ((1 << INAT_PFX_BITS) - 1) +-#define INAT_PFX_MASK (INAT_PFX_MAX << INAT_PFX_OFFS) +-/* Escape opcodes */ +-#define INAT_ESC_OFFS (INAT_PFX_OFFS + INAT_PFX_BITS) +-#define INAT_ESC_BITS 2 +-#define INAT_ESC_MAX ((1 << INAT_ESC_BITS) - 1) +-#define INAT_ESC_MASK (INAT_ESC_MAX << INAT_ESC_OFFS) +-/* Group opcodes (1-16) */ +-#define INAT_GRP_OFFS (INAT_ESC_OFFS + INAT_ESC_BITS) +-#define INAT_GRP_BITS 5 +-#define INAT_GRP_MAX ((1 << INAT_GRP_BITS) - 1) +-#define INAT_GRP_MASK (INAT_GRP_MAX << INAT_GRP_OFFS) +-/* Immediates */ +-#define INAT_IMM_OFFS (INAT_GRP_OFFS + INAT_GRP_BITS) +-#define INAT_IMM_BITS 3 +-#define INAT_IMM_MASK (((1 << INAT_IMM_BITS) - 1) << INAT_IMM_OFFS) +-/* Flags */ +-#define INAT_FLAG_OFFS (INAT_IMM_OFFS + INAT_IMM_BITS) +-#define INAT_MODRM (1 << (INAT_FLAG_OFFS)) +-#define INAT_FORCE64 (1 << (INAT_FLAG_OFFS + 1)) +-#define INAT_SCNDIMM (1 << (INAT_FLAG_OFFS + 2)) +-#define INAT_MOFFSET (1 << (INAT_FLAG_OFFS + 3)) +-#define INAT_VARIANT (1 << (INAT_FLAG_OFFS + 4)) +-#define INAT_VEXOK (1 << (INAT_FLAG_OFFS + 5)) +-#define INAT_VEXONLY (1 << (INAT_FLAG_OFFS + 6)) +-#define INAT_EVEXONLY (1 << (INAT_FLAG_OFFS + 7)) +-/* Attribute making macros for attribute tables */ +-#define INAT_MAKE_PREFIX(pfx) (pfx << INAT_PFX_OFFS) +-#define INAT_MAKE_ESCAPE(esc) (esc << INAT_ESC_OFFS) +-#define INAT_MAKE_GROUP(grp) ((grp << INAT_GRP_OFFS) | INAT_MODRM) +-#define INAT_MAKE_IMM(imm) (imm << INAT_IMM_OFFS) +- +-/* Attribute search APIs */ +-extern insn_attr_t inat_get_opcode_attribute(insn_byte_t opcode); +-extern int inat_get_last_prefix_id(insn_byte_t last_pfx); +-extern insn_attr_t inat_get_escape_attribute(insn_byte_t opcode, +- int lpfx_id, +- insn_attr_t esc_attr); +-extern insn_attr_t inat_get_group_attribute(insn_byte_t modrm, +- int lpfx_id, +- insn_attr_t esc_attr); +-extern insn_attr_t inat_get_avx_attribute(insn_byte_t opcode, +- insn_byte_t vex_m, +- insn_byte_t vex_pp); +- +-/* Attribute checking functions */ +-static inline int inat_is_legacy_prefix(insn_attr_t attr) +-{ +- attr &= INAT_PFX_MASK; +- return attr && attr <= INAT_LGCPFX_MAX; +-} +- +-static inline int inat_is_address_size_prefix(insn_attr_t attr) +-{ +- return (attr & INAT_PFX_MASK) == INAT_PFX_ADDRSZ; +-} +- +-static inline int inat_is_operand_size_prefix(insn_attr_t attr) +-{ +- return (attr & INAT_PFX_MASK) == INAT_PFX_OPNDSZ; +-} +- +-static inline int inat_is_rex_prefix(insn_attr_t attr) +-{ +- return (attr & INAT_PFX_MASK) == INAT_PFX_REX; +-} +- +-static inline int inat_last_prefix_id(insn_attr_t attr) +-{ +- if ((attr & INAT_PFX_MASK) > INAT_LSTPFX_MAX) +- return 0; +- else +- return attr & INAT_PFX_MASK; +-} +- +-static inline int inat_is_vex_prefix(insn_attr_t attr) +-{ +- attr &= INAT_PFX_MASK; +- return attr == INAT_PFX_VEX2 || attr == INAT_PFX_VEX3 || +- attr == INAT_PFX_EVEX; +-} +- +-static inline int inat_is_evex_prefix(insn_attr_t attr) +-{ +- return (attr & INAT_PFX_MASK) == INAT_PFX_EVEX; +-} +- +-static inline int inat_is_vex3_prefix(insn_attr_t attr) +-{ +- return (attr & INAT_PFX_MASK) == INAT_PFX_VEX3; +-} +- +-static inline int inat_is_escape(insn_attr_t attr) +-{ +- return attr & INAT_ESC_MASK; +-} +- +-static inline int inat_escape_id(insn_attr_t attr) +-{ +- return (attr & INAT_ESC_MASK) >> INAT_ESC_OFFS; +-} +- +-static inline int inat_is_group(insn_attr_t attr) +-{ +- return attr & INAT_GRP_MASK; +-} +- +-static inline int inat_group_id(insn_attr_t attr) +-{ +- return (attr & INAT_GRP_MASK) >> INAT_GRP_OFFS; +-} +- +-static inline int inat_group_common_attribute(insn_attr_t attr) +-{ +- return attr & ~INAT_GRP_MASK; +-} +- +-static inline int inat_has_immediate(insn_attr_t attr) +-{ +- return attr & INAT_IMM_MASK; +-} +- +-static inline int inat_immediate_size(insn_attr_t attr) +-{ +- return (attr & INAT_IMM_MASK) >> INAT_IMM_OFFS; +-} +- +-static inline int inat_has_modrm(insn_attr_t attr) +-{ +- return attr & INAT_MODRM; +-} +- +-static inline int inat_is_force64(insn_attr_t attr) +-{ +- return attr & INAT_FORCE64; +-} +- +-static inline int inat_has_second_immediate(insn_attr_t attr) +-{ +- return attr & INAT_SCNDIMM; +-} +- +-static inline int inat_has_moffset(insn_attr_t attr) +-{ +- return attr & INAT_MOFFSET; +-} +- +-static inline int inat_has_variant(insn_attr_t attr) +-{ +- return attr & INAT_VARIANT; +-} +- +-static inline int inat_accept_vex(insn_attr_t attr) +-{ +- return attr & INAT_VEXOK; +-} +- +-static inline int inat_must_vex(insn_attr_t attr) +-{ +- return attr & (INAT_VEXONLY | INAT_EVEXONLY); +-} +- +-static inline int inat_must_evex(insn_attr_t attr) +-{ +- return attr & INAT_EVEXONLY; +-} +-#endif +diff --git a/tools/objtool/arch/x86/insn/inat_types.h b/tools/objtool/arch/x86/insn/inat_types.h +deleted file mode 100644 +index cb3c20ce39cf..000000000000 +--- a/tools/objtool/arch/x86/insn/inat_types.h ++++ /dev/null +@@ -1,29 +0,0 @@ +-#ifndef _ASM_X86_INAT_TYPES_H +-#define _ASM_X86_INAT_TYPES_H +-/* +- * x86 instruction attributes +- * +- * Written by Masami Hiramatsu +- * +- * This program is free software; you can redistribute it and/or modify +- * it under the terms of the GNU General Public License as published by +- * the Free Software Foundation; either version 2 of the License, or +- * (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, write to the Free Software +- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +- * +- */ +- +-/* Instruction attributes */ +-typedef unsigned int insn_attr_t; +-typedef unsigned char insn_byte_t; +-typedef signed int insn_value_t; +- +-#endif +diff --git a/tools/objtool/arch/x86/insn/insn.c b/tools/objtool/arch/x86/insn/insn.c +deleted file mode 100644 +index ca983e2bea8b..000000000000 +--- a/tools/objtool/arch/x86/insn/insn.c ++++ /dev/null +@@ -1,606 +0,0 @@ +-/* +- * x86 instruction analysis +- * +- * This program is free software; you can redistribute it and/or modify +- * it under the terms of the GNU General Public License as published by +- * the Free Software Foundation; either version 2 of the License, or +- * (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, write to the Free Software +- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +- * +- * Copyright (C) IBM Corporation, 2002, 2004, 2009 +- */ +- +-#ifdef __KERNEL__ +-#include +-#else +-#include +-#endif +-#include "inat.h" +-#include "insn.h" +- +-/* Verify next sizeof(t) bytes can be on the same instruction */ +-#define validate_next(t, insn, n) \ +- ((insn)->next_byte + sizeof(t) + n <= (insn)->end_kaddr) +- +-#define __get_next(t, insn) \ +- ({ t r = *(t*)insn->next_byte; insn->next_byte += sizeof(t); r; }) +- +-#define __peek_nbyte_next(t, insn, n) \ +- ({ t r = *(t*)((insn)->next_byte + n); r; }) +- +-#define get_next(t, insn) \ +- ({ if (unlikely(!validate_next(t, insn, 0))) goto err_out; __get_next(t, insn); }) +- +-#define peek_nbyte_next(t, insn, n) \ +- ({ if (unlikely(!validate_next(t, insn, n))) goto err_out; __peek_nbyte_next(t, insn, n); }) +- +-#define peek_next(t, insn) peek_nbyte_next(t, insn, 0) +- +-/** +- * insn_init() - initialize struct insn +- * @insn: &struct insn to be initialized +- * @kaddr: address (in kernel memory) of instruction (or copy thereof) +- * @x86_64: !0 for 64-bit kernel or 64-bit app +- */ +-void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64) +-{ +- /* +- * Instructions longer than MAX_INSN_SIZE (15 bytes) are invalid +- * even if the input buffer is long enough to hold them. +- */ +- if (buf_len > MAX_INSN_SIZE) +- buf_len = MAX_INSN_SIZE; +- +- memset(insn, 0, sizeof(*insn)); +- insn->kaddr = kaddr; +- insn->end_kaddr = kaddr + buf_len; +- insn->next_byte = kaddr; +- insn->x86_64 = x86_64 ? 1 : 0; +- insn->opnd_bytes = 4; +- if (x86_64) +- insn->addr_bytes = 8; +- else +- insn->addr_bytes = 4; +-} +- +-/** +- * insn_get_prefixes - scan x86 instruction prefix bytes +- * @insn: &struct insn containing instruction +- * +- * Populates the @insn->prefixes bitmap, and updates @insn->next_byte +- * to point to the (first) opcode. No effect if @insn->prefixes.got +- * is already set. +- */ +-void insn_get_prefixes(struct insn *insn) +-{ +- struct insn_field *prefixes = &insn->prefixes; +- insn_attr_t attr; +- insn_byte_t b, lb; +- int i, nb; +- +- if (prefixes->got) +- return; +- +- nb = 0; +- lb = 0; +- b = peek_next(insn_byte_t, insn); +- attr = inat_get_opcode_attribute(b); +- while (inat_is_legacy_prefix(attr)) { +- /* Skip if same prefix */ +- for (i = 0; i < nb; i++) +- if (prefixes->bytes[i] == b) +- goto found; +- if (nb == 4) +- /* Invalid instruction */ +- break; +- prefixes->bytes[nb++] = b; +- if (inat_is_address_size_prefix(attr)) { +- /* address size switches 2/4 or 4/8 */ +- if (insn->x86_64) +- insn->addr_bytes ^= 12; +- else +- insn->addr_bytes ^= 6; +- } else if (inat_is_operand_size_prefix(attr)) { +- /* oprand size switches 2/4 */ +- insn->opnd_bytes ^= 6; +- } +-found: +- prefixes->nbytes++; +- insn->next_byte++; +- lb = b; +- b = peek_next(insn_byte_t, insn); +- attr = inat_get_opcode_attribute(b); +- } +- /* Set the last prefix */ +- if (lb && lb != insn->prefixes.bytes[3]) { +- if (unlikely(insn->prefixes.bytes[3])) { +- /* Swap the last prefix */ +- b = insn->prefixes.bytes[3]; +- for (i = 0; i < nb; i++) +- if (prefixes->bytes[i] == lb) +- prefixes->bytes[i] = b; +- } +- insn->prefixes.bytes[3] = lb; +- } +- +- /* Decode REX prefix */ +- if (insn->x86_64) { +- b = peek_next(insn_byte_t, insn); +- attr = inat_get_opcode_attribute(b); +- if (inat_is_rex_prefix(attr)) { +- insn->rex_prefix.value = b; +- insn->rex_prefix.nbytes = 1; +- insn->next_byte++; +- if (X86_REX_W(b)) +- /* REX.W overrides opnd_size */ +- insn->opnd_bytes = 8; +- } +- } +- insn->rex_prefix.got = 1; +- +- /* Decode VEX prefix */ +- b = peek_next(insn_byte_t, insn); +- attr = inat_get_opcode_attribute(b); +- if (inat_is_vex_prefix(attr)) { +- insn_byte_t b2 = peek_nbyte_next(insn_byte_t, insn, 1); +- if (!insn->x86_64) { +- /* +- * In 32-bits mode, if the [7:6] bits (mod bits of +- * ModRM) on the second byte are not 11b, it is +- * LDS or LES or BOUND. +- */ +- if (X86_MODRM_MOD(b2) != 3) +- goto vex_end; +- } +- insn->vex_prefix.bytes[0] = b; +- insn->vex_prefix.bytes[1] = b2; +- if (inat_is_evex_prefix(attr)) { +- b2 = peek_nbyte_next(insn_byte_t, insn, 2); +- insn->vex_prefix.bytes[2] = b2; +- b2 = peek_nbyte_next(insn_byte_t, insn, 3); +- insn->vex_prefix.bytes[3] = b2; +- insn->vex_prefix.nbytes = 4; +- insn->next_byte += 4; +- if (insn->x86_64 && X86_VEX_W(b2)) +- /* VEX.W overrides opnd_size */ +- insn->opnd_bytes = 8; +- } else if (inat_is_vex3_prefix(attr)) { +- b2 = peek_nbyte_next(insn_byte_t, insn, 2); +- insn->vex_prefix.bytes[2] = b2; +- insn->vex_prefix.nbytes = 3; +- insn->next_byte += 3; +- if (insn->x86_64 && X86_VEX_W(b2)) +- /* VEX.W overrides opnd_size */ +- insn->opnd_bytes = 8; +- } else { +- /* +- * For VEX2, fake VEX3-like byte#2. +- * Makes it easier to decode vex.W, vex.vvvv, +- * vex.L and vex.pp. Masking with 0x7f sets vex.W == 0. +- */ +- insn->vex_prefix.bytes[2] = b2 & 0x7f; +- insn->vex_prefix.nbytes = 2; +- insn->next_byte += 2; +- } +- } +-vex_end: +- insn->vex_prefix.got = 1; +- +- prefixes->got = 1; +- +-err_out: +- return; +-} +- +-/** +- * insn_get_opcode - collect opcode(s) +- * @insn: &struct insn containing instruction +- * +- * Populates @insn->opcode, updates @insn->next_byte to point past the +- * opcode byte(s), and set @insn->attr (except for groups). +- * If necessary, first collects any preceding (prefix) bytes. +- * Sets @insn->opcode.value = opcode1. No effect if @insn->opcode.got +- * is already 1. +- */ +-void insn_get_opcode(struct insn *insn) +-{ +- struct insn_field *opcode = &insn->opcode; +- insn_byte_t op; +- int pfx_id; +- if (opcode->got) +- return; +- if (!insn->prefixes.got) +- insn_get_prefixes(insn); +- +- /* Get first opcode */ +- op = get_next(insn_byte_t, insn); +- opcode->bytes[0] = op; +- opcode->nbytes = 1; +- +- /* Check if there is VEX prefix or not */ +- if (insn_is_avx(insn)) { +- insn_byte_t m, p; +- m = insn_vex_m_bits(insn); +- p = insn_vex_p_bits(insn); +- insn->attr = inat_get_avx_attribute(op, m, p); +- if ((inat_must_evex(insn->attr) && !insn_is_evex(insn)) || +- (!inat_accept_vex(insn->attr) && +- !inat_is_group(insn->attr))) +- insn->attr = 0; /* This instruction is bad */ +- goto end; /* VEX has only 1 byte for opcode */ +- } +- +- insn->attr = inat_get_opcode_attribute(op); +- while (inat_is_escape(insn->attr)) { +- /* Get escaped opcode */ +- op = get_next(insn_byte_t, insn); +- opcode->bytes[opcode->nbytes++] = op; +- pfx_id = insn_last_prefix_id(insn); +- insn->attr = inat_get_escape_attribute(op, pfx_id, insn->attr); +- } +- if (inat_must_vex(insn->attr)) +- insn->attr = 0; /* This instruction is bad */ +-end: +- opcode->got = 1; +- +-err_out: +- return; +-} +- +-/** +- * insn_get_modrm - collect ModRM byte, if any +- * @insn: &struct insn containing instruction +- * +- * Populates @insn->modrm and updates @insn->next_byte to point past the +- * ModRM byte, if any. If necessary, first collects the preceding bytes +- * (prefixes and opcode(s)). No effect if @insn->modrm.got is already 1. +- */ +-void insn_get_modrm(struct insn *insn) +-{ +- struct insn_field *modrm = &insn->modrm; +- insn_byte_t pfx_id, mod; +- if (modrm->got) +- return; +- if (!insn->opcode.got) +- insn_get_opcode(insn); +- +- if (inat_has_modrm(insn->attr)) { +- mod = get_next(insn_byte_t, insn); +- modrm->value = mod; +- modrm->nbytes = 1; +- if (inat_is_group(insn->attr)) { +- pfx_id = insn_last_prefix_id(insn); +- insn->attr = inat_get_group_attribute(mod, pfx_id, +- insn->attr); +- if (insn_is_avx(insn) && !inat_accept_vex(insn->attr)) +- insn->attr = 0; /* This is bad */ +- } +- } +- +- if (insn->x86_64 && inat_is_force64(insn->attr)) +- insn->opnd_bytes = 8; +- modrm->got = 1; +- +-err_out: +- return; +-} +- +- +-/** +- * insn_rip_relative() - Does instruction use RIP-relative addressing mode? +- * @insn: &struct insn containing instruction +- * +- * If necessary, first collects the instruction up to and including the +- * ModRM byte. No effect if @insn->x86_64 is 0. +- */ +-int insn_rip_relative(struct insn *insn) +-{ +- struct insn_field *modrm = &insn->modrm; +- +- if (!insn->x86_64) +- return 0; +- if (!modrm->got) +- insn_get_modrm(insn); +- /* +- * For rip-relative instructions, the mod field (top 2 bits) +- * is zero and the r/m field (bottom 3 bits) is 0x5. +- */ +- return (modrm->nbytes && (modrm->value & 0xc7) == 0x5); +-} +- +-/** +- * insn_get_sib() - Get the SIB byte of instruction +- * @insn: &struct insn containing instruction +- * +- * If necessary, first collects the instruction up to and including the +- * ModRM byte. +- */ +-void insn_get_sib(struct insn *insn) +-{ +- insn_byte_t modrm; +- +- if (insn->sib.got) +- return; +- if (!insn->modrm.got) +- insn_get_modrm(insn); +- if (insn->modrm.nbytes) { +- modrm = (insn_byte_t)insn->modrm.value; +- if (insn->addr_bytes != 2 && +- X86_MODRM_MOD(modrm) != 3 && X86_MODRM_RM(modrm) == 4) { +- insn->sib.value = get_next(insn_byte_t, insn); +- insn->sib.nbytes = 1; +- } +- } +- insn->sib.got = 1; +- +-err_out: +- return; +-} +- +- +-/** +- * insn_get_displacement() - Get the displacement of instruction +- * @insn: &struct insn containing instruction +- * +- * If necessary, first collects the instruction up to and including the +- * SIB byte. +- * Displacement value is sign-expanded. +- */ +-void insn_get_displacement(struct insn *insn) +-{ +- insn_byte_t mod, rm, base; +- +- if (insn->displacement.got) +- return; +- if (!insn->sib.got) +- insn_get_sib(insn); +- if (insn->modrm.nbytes) { +- /* +- * Interpreting the modrm byte: +- * mod = 00 - no displacement fields (exceptions below) +- * mod = 01 - 1-byte displacement field +- * mod = 10 - displacement field is 4 bytes, or 2 bytes if +- * address size = 2 (0x67 prefix in 32-bit mode) +- * mod = 11 - no memory operand +- * +- * If address size = 2... +- * mod = 00, r/m = 110 - displacement field is 2 bytes +- * +- * If address size != 2... +- * mod != 11, r/m = 100 - SIB byte exists +- * mod = 00, SIB base = 101 - displacement field is 4 bytes +- * mod = 00, r/m = 101 - rip-relative addressing, displacement +- * field is 4 bytes +- */ +- mod = X86_MODRM_MOD(insn->modrm.value); +- rm = X86_MODRM_RM(insn->modrm.value); +- base = X86_SIB_BASE(insn->sib.value); +- if (mod == 3) +- goto out; +- if (mod == 1) { +- insn->displacement.value = get_next(signed char, insn); +- insn->displacement.nbytes = 1; +- } else if (insn->addr_bytes == 2) { +- if ((mod == 0 && rm == 6) || mod == 2) { +- insn->displacement.value = +- get_next(short, insn); +- insn->displacement.nbytes = 2; +- } +- } else { +- if ((mod == 0 && rm == 5) || mod == 2 || +- (mod == 0 && base == 5)) { +- insn->displacement.value = get_next(int, insn); +- insn->displacement.nbytes = 4; +- } +- } +- } +-out: +- insn->displacement.got = 1; +- +-err_out: +- return; +-} +- +-/* Decode moffset16/32/64. Return 0 if failed */ +-static int __get_moffset(struct insn *insn) +-{ +- switch (insn->addr_bytes) { +- case 2: +- insn->moffset1.value = get_next(short, insn); +- insn->moffset1.nbytes = 2; +- break; +- case 4: +- insn->moffset1.value = get_next(int, insn); +- insn->moffset1.nbytes = 4; +- break; +- case 8: +- insn->moffset1.value = get_next(int, insn); +- insn->moffset1.nbytes = 4; +- insn->moffset2.value = get_next(int, insn); +- insn->moffset2.nbytes = 4; +- break; +- default: /* opnd_bytes must be modified manually */ +- goto err_out; +- } +- insn->moffset1.got = insn->moffset2.got = 1; +- +- return 1; +- +-err_out: +- return 0; +-} +- +-/* Decode imm v32(Iz). Return 0 if failed */ +-static int __get_immv32(struct insn *insn) +-{ +- switch (insn->opnd_bytes) { +- case 2: +- insn->immediate.value = get_next(short, insn); +- insn->immediate.nbytes = 2; +- break; +- case 4: +- case 8: +- insn->immediate.value = get_next(int, insn); +- insn->immediate.nbytes = 4; +- break; +- default: /* opnd_bytes must be modified manually */ +- goto err_out; +- } +- +- return 1; +- +-err_out: +- return 0; +-} +- +-/* Decode imm v64(Iv/Ov), Return 0 if failed */ +-static int __get_immv(struct insn *insn) +-{ +- switch (insn->opnd_bytes) { +- case 2: +- insn->immediate1.value = get_next(short, insn); +- insn->immediate1.nbytes = 2; +- break; +- case 4: +- insn->immediate1.value = get_next(int, insn); +- insn->immediate1.nbytes = 4; +- break; +- case 8: +- insn->immediate1.value = get_next(int, insn); +- insn->immediate1.nbytes = 4; +- insn->immediate2.value = get_next(int, insn); +- insn->immediate2.nbytes = 4; +- break; +- default: /* opnd_bytes must be modified manually */ +- goto err_out; +- } +- insn->immediate1.got = insn->immediate2.got = 1; +- +- return 1; +-err_out: +- return 0; +-} +- +-/* Decode ptr16:16/32(Ap) */ +-static int __get_immptr(struct insn *insn) +-{ +- switch (insn->opnd_bytes) { +- case 2: +- insn->immediate1.value = get_next(short, insn); +- insn->immediate1.nbytes = 2; +- break; +- case 4: +- insn->immediate1.value = get_next(int, insn); +- insn->immediate1.nbytes = 4; +- break; +- case 8: +- /* ptr16:64 is not exist (no segment) */ +- return 0; +- default: /* opnd_bytes must be modified manually */ +- goto err_out; +- } +- insn->immediate2.value = get_next(unsigned short, insn); +- insn->immediate2.nbytes = 2; +- insn->immediate1.got = insn->immediate2.got = 1; +- +- return 1; +-err_out: +- return 0; +-} +- +-/** +- * insn_get_immediate() - Get the immediates of instruction +- * @insn: &struct insn containing instruction +- * +- * If necessary, first collects the instruction up to and including the +- * displacement bytes. +- * Basically, most of immediates are sign-expanded. Unsigned-value can be +- * get by bit masking with ((1 << (nbytes * 8)) - 1) +- */ +-void insn_get_immediate(struct insn *insn) +-{ +- if (insn->immediate.got) +- return; +- if (!insn->displacement.got) +- insn_get_displacement(insn); +- +- if (inat_has_moffset(insn->attr)) { +- if (!__get_moffset(insn)) +- goto err_out; +- goto done; +- } +- +- if (!inat_has_immediate(insn->attr)) +- /* no immediates */ +- goto done; +- +- switch (inat_immediate_size(insn->attr)) { +- case INAT_IMM_BYTE: +- insn->immediate.value = get_next(signed char, insn); +- insn->immediate.nbytes = 1; +- break; +- case INAT_IMM_WORD: +- insn->immediate.value = get_next(short, insn); +- insn->immediate.nbytes = 2; +- break; +- case INAT_IMM_DWORD: +- insn->immediate.value = get_next(int, insn); +- insn->immediate.nbytes = 4; +- break; +- case INAT_IMM_QWORD: +- insn->immediate1.value = get_next(int, insn); +- insn->immediate1.nbytes = 4; +- insn->immediate2.value = get_next(int, insn); +- insn->immediate2.nbytes = 4; +- break; +- case INAT_IMM_PTR: +- if (!__get_immptr(insn)) +- goto err_out; +- break; +- case INAT_IMM_VWORD32: +- if (!__get_immv32(insn)) +- goto err_out; +- break; +- case INAT_IMM_VWORD: +- if (!__get_immv(insn)) +- goto err_out; +- break; +- default: +- /* Here, insn must have an immediate, but failed */ +- goto err_out; +- } +- if (inat_has_second_immediate(insn->attr)) { +- insn->immediate2.value = get_next(signed char, insn); +- insn->immediate2.nbytes = 1; +- } +-done: +- insn->immediate.got = 1; +- +-err_out: +- return; +-} +- +-/** +- * insn_get_length() - Get the length of instruction +- * @insn: &struct insn containing instruction +- * +- * If necessary, first collects the instruction up to and including the +- * immediates bytes. +- */ +-void insn_get_length(struct insn *insn) +-{ +- if (insn->length) +- return; +- if (!insn->immediate.got) +- insn_get_immediate(insn); +- insn->length = (unsigned char)((unsigned long)insn->next_byte +- - (unsigned long)insn->kaddr); +-} +diff --git a/tools/objtool/arch/x86/insn/insn.h b/tools/objtool/arch/x86/insn/insn.h +deleted file mode 100644 +index e23578c7b1be..000000000000 +--- a/tools/objtool/arch/x86/insn/insn.h ++++ /dev/null +@@ -1,211 +0,0 @@ +-#ifndef _ASM_X86_INSN_H +-#define _ASM_X86_INSN_H +-/* +- * x86 instruction analysis +- * +- * This program is free software; you can redistribute it and/or modify +- * it under the terms of the GNU General Public License as published by +- * the Free Software Foundation; either version 2 of the License, or +- * (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, write to the Free Software +- * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. +- * +- * Copyright (C) IBM Corporation, 2009 +- */ +- +-/* insn_attr_t is defined in inat.h */ +-#include "inat.h" +- +-struct insn_field { +- union { +- insn_value_t value; +- insn_byte_t bytes[4]; +- }; +- /* !0 if we've run insn_get_xxx() for this field */ +- unsigned char got; +- unsigned char nbytes; +-}; +- +-struct insn { +- struct insn_field prefixes; /* +- * Prefixes +- * prefixes.bytes[3]: last prefix +- */ +- struct insn_field rex_prefix; /* REX prefix */ +- struct insn_field vex_prefix; /* VEX prefix */ +- struct insn_field opcode; /* +- * opcode.bytes[0]: opcode1 +- * opcode.bytes[1]: opcode2 +- * opcode.bytes[2]: opcode3 +- */ +- struct insn_field modrm; +- struct insn_field sib; +- struct insn_field displacement; +- union { +- struct insn_field immediate; +- struct insn_field moffset1; /* for 64bit MOV */ +- struct insn_field immediate1; /* for 64bit imm or off16/32 */ +- }; +- union { +- struct insn_field moffset2; /* for 64bit MOV */ +- struct insn_field immediate2; /* for 64bit imm or seg16 */ +- }; +- +- insn_attr_t attr; +- unsigned char opnd_bytes; +- unsigned char addr_bytes; +- unsigned char length; +- unsigned char x86_64; +- +- const insn_byte_t *kaddr; /* kernel address of insn to analyze */ +- const insn_byte_t *end_kaddr; /* kernel address of last insn in buffer */ +- const insn_byte_t *next_byte; +-}; +- +-#define MAX_INSN_SIZE 15 +- +-#define X86_MODRM_MOD(modrm) (((modrm) & 0xc0) >> 6) +-#define X86_MODRM_REG(modrm) (((modrm) & 0x38) >> 3) +-#define X86_MODRM_RM(modrm) ((modrm) & 0x07) +- +-#define X86_SIB_SCALE(sib) (((sib) & 0xc0) >> 6) +-#define X86_SIB_INDEX(sib) (((sib) & 0x38) >> 3) +-#define X86_SIB_BASE(sib) ((sib) & 0x07) +- +-#define X86_REX_W(rex) ((rex) & 8) +-#define X86_REX_R(rex) ((rex) & 4) +-#define X86_REX_X(rex) ((rex) & 2) +-#define X86_REX_B(rex) ((rex) & 1) +- +-/* VEX bit flags */ +-#define X86_VEX_W(vex) ((vex) & 0x80) /* VEX3 Byte2 */ +-#define X86_VEX_R(vex) ((vex) & 0x80) /* VEX2/3 Byte1 */ +-#define X86_VEX_X(vex) ((vex) & 0x40) /* VEX3 Byte1 */ +-#define X86_VEX_B(vex) ((vex) & 0x20) /* VEX3 Byte1 */ +-#define X86_VEX_L(vex) ((vex) & 0x04) /* VEX3 Byte2, VEX2 Byte1 */ +-/* VEX bit fields */ +-#define X86_EVEX_M(vex) ((vex) & 0x03) /* EVEX Byte1 */ +-#define X86_VEX3_M(vex) ((vex) & 0x1f) /* VEX3 Byte1 */ +-#define X86_VEX2_M 1 /* VEX2.M always 1 */ +-#define X86_VEX_V(vex) (((vex) & 0x78) >> 3) /* VEX3 Byte2, VEX2 Byte1 */ +-#define X86_VEX_P(vex) ((vex) & 0x03) /* VEX3 Byte2, VEX2 Byte1 */ +-#define X86_VEX_M_MAX 0x1f /* VEX3.M Maximum value */ +- +-extern void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64); +-extern void insn_get_prefixes(struct insn *insn); +-extern void insn_get_opcode(struct insn *insn); +-extern void insn_get_modrm(struct insn *insn); +-extern void insn_get_sib(struct insn *insn); +-extern void insn_get_displacement(struct insn *insn); +-extern void insn_get_immediate(struct insn *insn); +-extern void insn_get_length(struct insn *insn); +- +-/* Attribute will be determined after getting ModRM (for opcode groups) */ +-static inline void insn_get_attribute(struct insn *insn) +-{ +- insn_get_modrm(insn); +-} +- +-/* Instruction uses RIP-relative addressing */ +-extern int insn_rip_relative(struct insn *insn); +- +-/* Init insn for kernel text */ +-static inline void kernel_insn_init(struct insn *insn, +- const void *kaddr, int buf_len) +-{ +-#ifdef CONFIG_X86_64 +- insn_init(insn, kaddr, buf_len, 1); +-#else /* CONFIG_X86_32 */ +- insn_init(insn, kaddr, buf_len, 0); +-#endif +-} +- +-static inline int insn_is_avx(struct insn *insn) +-{ +- if (!insn->prefixes.got) +- insn_get_prefixes(insn); +- return (insn->vex_prefix.value != 0); +-} +- +-static inline int insn_is_evex(struct insn *insn) +-{ +- if (!insn->prefixes.got) +- insn_get_prefixes(insn); +- return (insn->vex_prefix.nbytes == 4); +-} +- +-/* Ensure this instruction is decoded completely */ +-static inline int insn_complete(struct insn *insn) +-{ +- return insn->opcode.got && insn->modrm.got && insn->sib.got && +- insn->displacement.got && insn->immediate.got; +-} +- +-static inline insn_byte_t insn_vex_m_bits(struct insn *insn) +-{ +- if (insn->vex_prefix.nbytes == 2) /* 2 bytes VEX */ +- return X86_VEX2_M; +- else if (insn->vex_prefix.nbytes == 3) /* 3 bytes VEX */ +- return X86_VEX3_M(insn->vex_prefix.bytes[1]); +- else /* EVEX */ +- return X86_EVEX_M(insn->vex_prefix.bytes[1]); +-} +- +-static inline insn_byte_t insn_vex_p_bits(struct insn *insn) +-{ +- if (insn->vex_prefix.nbytes == 2) /* 2 bytes VEX */ +- return X86_VEX_P(insn->vex_prefix.bytes[1]); +- else +- return X86_VEX_P(insn->vex_prefix.bytes[2]); +-} +- +-/* Get the last prefix id from last prefix or VEX prefix */ +-static inline int insn_last_prefix_id(struct insn *insn) +-{ +- if (insn_is_avx(insn)) +- return insn_vex_p_bits(insn); /* VEX_p is a SIMD prefix id */ +- +- if (insn->prefixes.bytes[3]) +- return inat_get_last_prefix_id(insn->prefixes.bytes[3]); +- +- return 0; +-} +- +-/* Offset of each field from kaddr */ +-static inline int insn_offset_rex_prefix(struct insn *insn) +-{ +- return insn->prefixes.nbytes; +-} +-static inline int insn_offset_vex_prefix(struct insn *insn) +-{ +- return insn_offset_rex_prefix(insn) + insn->rex_prefix.nbytes; +-} +-static inline int insn_offset_opcode(struct insn *insn) +-{ +- return insn_offset_vex_prefix(insn) + insn->vex_prefix.nbytes; +-} +-static inline int insn_offset_modrm(struct insn *insn) +-{ +- return insn_offset_opcode(insn) + insn->opcode.nbytes; +-} +-static inline int insn_offset_sib(struct insn *insn) +-{ +- return insn_offset_modrm(insn) + insn->modrm.nbytes; +-} +-static inline int insn_offset_displacement(struct insn *insn) +-{ +- return insn_offset_sib(insn) + insn->sib.nbytes; +-} +-static inline int insn_offset_immediate(struct insn *insn) +-{ +- return insn_offset_displacement(insn) + insn->displacement.nbytes; +-} +- +-#endif /* _ASM_X86_INSN_H */ +diff --git a/tools/objtool/arch/x86/insn/x86-opcode-map.txt b/tools/objtool/arch/x86/insn/x86-opcode-map.txt +deleted file mode 100644 +index 12e377184ee4..000000000000 +--- a/tools/objtool/arch/x86/insn/x86-opcode-map.txt ++++ /dev/null +@@ -1,1063 +0,0 @@ +-# x86 Opcode Maps +-# +-# This is (mostly) based on following documentations. +-# - Intel(R) 64 and IA-32 Architectures Software Developer's Manual Vol.2C +-# (#326018-047US, June 2013) +-# +-# +-# Table: table-name +-# Referrer: escaped-name +-# AVXcode: avx-code +-# opcode: mnemonic|GrpXXX [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...] +-# (or) +-# opcode: escape # escaped-name +-# EndTable +-# +-# mnemonics that begin with lowercase 'v' accept a VEX or EVEX prefix +-# mnemonics that begin with lowercase 'k' accept a VEX prefix +-# +-# +-# GrpTable: GrpXXX +-# reg: mnemonic [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...] +-# EndTable +-# +-# AVX Superscripts +-# (ev): this opcode requires EVEX prefix. +-# (evo): this opcode is changed by EVEX prefix (EVEX opcode) +-# (v): this opcode requires VEX prefix. +-# (v1): this opcode only supports 128bit VEX. +-# +-# Last Prefix Superscripts +-# - (66): the last prefix is 0x66 +-# - (F3): the last prefix is 0xF3 +-# - (F2): the last prefix is 0xF2 +-# - (!F3) : the last prefix is not 0xF3 (including non-last prefix case) +-# - (66&F2): Both 0x66 and 0xF2 prefixes are specified. +- +-Table: one byte opcode +-Referrer: +-AVXcode: +-# 0x00 - 0x0f +-00: ADD Eb,Gb +-01: ADD Ev,Gv +-02: ADD Gb,Eb +-03: ADD Gv,Ev +-04: ADD AL,Ib +-05: ADD rAX,Iz +-06: PUSH ES (i64) +-07: POP ES (i64) +-08: OR Eb,Gb +-09: OR Ev,Gv +-0a: OR Gb,Eb +-0b: OR Gv,Ev +-0c: OR AL,Ib +-0d: OR rAX,Iz +-0e: PUSH CS (i64) +-0f: escape # 2-byte escape +-# 0x10 - 0x1f +-10: ADC Eb,Gb +-11: ADC Ev,Gv +-12: ADC Gb,Eb +-13: ADC Gv,Ev +-14: ADC AL,Ib +-15: ADC rAX,Iz +-16: PUSH SS (i64) +-17: POP SS (i64) +-18: SBB Eb,Gb +-19: SBB Ev,Gv +-1a: SBB Gb,Eb +-1b: SBB Gv,Ev +-1c: SBB AL,Ib +-1d: SBB rAX,Iz +-1e: PUSH DS (i64) +-1f: POP DS (i64) +-# 0x20 - 0x2f +-20: AND Eb,Gb +-21: AND Ev,Gv +-22: AND Gb,Eb +-23: AND Gv,Ev +-24: AND AL,Ib +-25: AND rAx,Iz +-26: SEG=ES (Prefix) +-27: DAA (i64) +-28: SUB Eb,Gb +-29: SUB Ev,Gv +-2a: SUB Gb,Eb +-2b: SUB Gv,Ev +-2c: SUB AL,Ib +-2d: SUB rAX,Iz +-2e: SEG=CS (Prefix) +-2f: DAS (i64) +-# 0x30 - 0x3f +-30: XOR Eb,Gb +-31: XOR Ev,Gv +-32: XOR Gb,Eb +-33: XOR Gv,Ev +-34: XOR AL,Ib +-35: XOR rAX,Iz +-36: SEG=SS (Prefix) +-37: AAA (i64) +-38: CMP Eb,Gb +-39: CMP Ev,Gv +-3a: CMP Gb,Eb +-3b: CMP Gv,Ev +-3c: CMP AL,Ib +-3d: CMP rAX,Iz +-3e: SEG=DS (Prefix) +-3f: AAS (i64) +-# 0x40 - 0x4f +-40: INC eAX (i64) | REX (o64) +-41: INC eCX (i64) | REX.B (o64) +-42: INC eDX (i64) | REX.X (o64) +-43: INC eBX (i64) | REX.XB (o64) +-44: INC eSP (i64) | REX.R (o64) +-45: INC eBP (i64) | REX.RB (o64) +-46: INC eSI (i64) | REX.RX (o64) +-47: INC eDI (i64) | REX.RXB (o64) +-48: DEC eAX (i64) | REX.W (o64) +-49: DEC eCX (i64) | REX.WB (o64) +-4a: DEC eDX (i64) | REX.WX (o64) +-4b: DEC eBX (i64) | REX.WXB (o64) +-4c: DEC eSP (i64) | REX.WR (o64) +-4d: DEC eBP (i64) | REX.WRB (o64) +-4e: DEC eSI (i64) | REX.WRX (o64) +-4f: DEC eDI (i64) | REX.WRXB (o64) +-# 0x50 - 0x5f +-50: PUSH rAX/r8 (d64) +-51: PUSH rCX/r9 (d64) +-52: PUSH rDX/r10 (d64) +-53: PUSH rBX/r11 (d64) +-54: PUSH rSP/r12 (d64) +-55: PUSH rBP/r13 (d64) +-56: PUSH rSI/r14 (d64) +-57: PUSH rDI/r15 (d64) +-58: POP rAX/r8 (d64) +-59: POP rCX/r9 (d64) +-5a: POP rDX/r10 (d64) +-5b: POP rBX/r11 (d64) +-5c: POP rSP/r12 (d64) +-5d: POP rBP/r13 (d64) +-5e: POP rSI/r14 (d64) +-5f: POP rDI/r15 (d64) +-# 0x60 - 0x6f +-60: PUSHA/PUSHAD (i64) +-61: POPA/POPAD (i64) +-62: BOUND Gv,Ma (i64) | EVEX (Prefix) +-63: ARPL Ew,Gw (i64) | MOVSXD Gv,Ev (o64) +-64: SEG=FS (Prefix) +-65: SEG=GS (Prefix) +-66: Operand-Size (Prefix) +-67: Address-Size (Prefix) +-68: PUSH Iz (d64) +-69: IMUL Gv,Ev,Iz +-6a: PUSH Ib (d64) +-6b: IMUL Gv,Ev,Ib +-6c: INS/INSB Yb,DX +-6d: INS/INSW/INSD Yz,DX +-6e: OUTS/OUTSB DX,Xb +-6f: OUTS/OUTSW/OUTSD DX,Xz +-# 0x70 - 0x7f +-70: JO Jb +-71: JNO Jb +-72: JB/JNAE/JC Jb +-73: JNB/JAE/JNC Jb +-74: JZ/JE Jb +-75: JNZ/JNE Jb +-76: JBE/JNA Jb +-77: JNBE/JA Jb +-78: JS Jb +-79: JNS Jb +-7a: JP/JPE Jb +-7b: JNP/JPO Jb +-7c: JL/JNGE Jb +-7d: JNL/JGE Jb +-7e: JLE/JNG Jb +-7f: JNLE/JG Jb +-# 0x80 - 0x8f +-80: Grp1 Eb,Ib (1A) +-81: Grp1 Ev,Iz (1A) +-82: Grp1 Eb,Ib (1A),(i64) +-83: Grp1 Ev,Ib (1A) +-84: TEST Eb,Gb +-85: TEST Ev,Gv +-86: XCHG Eb,Gb +-87: XCHG Ev,Gv +-88: MOV Eb,Gb +-89: MOV Ev,Gv +-8a: MOV Gb,Eb +-8b: MOV Gv,Ev +-8c: MOV Ev,Sw +-8d: LEA Gv,M +-8e: MOV Sw,Ew +-8f: Grp1A (1A) | POP Ev (d64) +-# 0x90 - 0x9f +-90: NOP | PAUSE (F3) | XCHG r8,rAX +-91: XCHG rCX/r9,rAX +-92: XCHG rDX/r10,rAX +-93: XCHG rBX/r11,rAX +-94: XCHG rSP/r12,rAX +-95: XCHG rBP/r13,rAX +-96: XCHG rSI/r14,rAX +-97: XCHG rDI/r15,rAX +-98: CBW/CWDE/CDQE +-99: CWD/CDQ/CQO +-9a: CALLF Ap (i64) +-9b: FWAIT/WAIT +-9c: PUSHF/D/Q Fv (d64) +-9d: POPF/D/Q Fv (d64) +-9e: SAHF +-9f: LAHF +-# 0xa0 - 0xaf +-a0: MOV AL,Ob +-a1: MOV rAX,Ov +-a2: MOV Ob,AL +-a3: MOV Ov,rAX +-a4: MOVS/B Yb,Xb +-a5: MOVS/W/D/Q Yv,Xv +-a6: CMPS/B Xb,Yb +-a7: CMPS/W/D Xv,Yv +-a8: TEST AL,Ib +-a9: TEST rAX,Iz +-aa: STOS/B Yb,AL +-ab: STOS/W/D/Q Yv,rAX +-ac: LODS/B AL,Xb +-ad: LODS/W/D/Q rAX,Xv +-ae: SCAS/B AL,Yb +-# Note: The May 2011 Intel manual shows Xv for the second parameter of the +-# next instruction but Yv is correct +-af: SCAS/W/D/Q rAX,Yv +-# 0xb0 - 0xbf +-b0: MOV AL/R8L,Ib +-b1: MOV CL/R9L,Ib +-b2: MOV DL/R10L,Ib +-b3: MOV BL/R11L,Ib +-b4: MOV AH/R12L,Ib +-b5: MOV CH/R13L,Ib +-b6: MOV DH/R14L,Ib +-b7: MOV BH/R15L,Ib +-b8: MOV rAX/r8,Iv +-b9: MOV rCX/r9,Iv +-ba: MOV rDX/r10,Iv +-bb: MOV rBX/r11,Iv +-bc: MOV rSP/r12,Iv +-bd: MOV rBP/r13,Iv +-be: MOV rSI/r14,Iv +-bf: MOV rDI/r15,Iv +-# 0xc0 - 0xcf +-c0: Grp2 Eb,Ib (1A) +-c1: Grp2 Ev,Ib (1A) +-c2: RETN Iw (f64) +-c3: RETN +-c4: LES Gz,Mp (i64) | VEX+2byte (Prefix) +-c5: LDS Gz,Mp (i64) | VEX+1byte (Prefix) +-c6: Grp11A Eb,Ib (1A) +-c7: Grp11B Ev,Iz (1A) +-c8: ENTER Iw,Ib +-c9: LEAVE (d64) +-ca: RETF Iw +-cb: RETF +-cc: INT3 +-cd: INT Ib +-ce: INTO (i64) +-cf: IRET/D/Q +-# 0xd0 - 0xdf +-d0: Grp2 Eb,1 (1A) +-d1: Grp2 Ev,1 (1A) +-d2: Grp2 Eb,CL (1A) +-d3: Grp2 Ev,CL (1A) +-d4: AAM Ib (i64) +-d5: AAD Ib (i64) +-d6: +-d7: XLAT/XLATB +-d8: ESC +-d9: ESC +-da: ESC +-db: ESC +-dc: ESC +-dd: ESC +-de: ESC +-df: ESC +-# 0xe0 - 0xef +-# Note: "forced64" is Intel CPU behavior: they ignore 0x66 prefix +-# in 64-bit mode. AMD CPUs accept 0x66 prefix, it causes RIP truncation +-# to 16 bits. In 32-bit mode, 0x66 is accepted by both Intel and AMD. +-e0: LOOPNE/LOOPNZ Jb (f64) +-e1: LOOPE/LOOPZ Jb (f64) +-e2: LOOP Jb (f64) +-e3: JrCXZ Jb (f64) +-e4: IN AL,Ib +-e5: IN eAX,Ib +-e6: OUT Ib,AL +-e7: OUT Ib,eAX +-# With 0x66 prefix in 64-bit mode, for AMD CPUs immediate offset +-# in "near" jumps and calls is 16-bit. For CALL, +-# push of return address is 16-bit wide, RSP is decremented by 2 +-# but is not truncated to 16 bits, unlike RIP. +-e8: CALL Jz (f64) +-e9: JMP-near Jz (f64) +-ea: JMP-far Ap (i64) +-eb: JMP-short Jb (f64) +-ec: IN AL,DX +-ed: IN eAX,DX +-ee: OUT DX,AL +-ef: OUT DX,eAX +-# 0xf0 - 0xff +-f0: LOCK (Prefix) +-f1: +-f2: REPNE (Prefix) | XACQUIRE (Prefix) +-f3: REP/REPE (Prefix) | XRELEASE (Prefix) +-f4: HLT +-f5: CMC +-f6: Grp3_1 Eb (1A) +-f7: Grp3_2 Ev (1A) +-f8: CLC +-f9: STC +-fa: CLI +-fb: STI +-fc: CLD +-fd: STD +-fe: Grp4 (1A) +-ff: Grp5 (1A) +-EndTable +- +-Table: 2-byte opcode (0x0f) +-Referrer: 2-byte escape +-AVXcode: 1 +-# 0x0f 0x00-0x0f +-00: Grp6 (1A) +-01: Grp7 (1A) +-02: LAR Gv,Ew +-03: LSL Gv,Ew +-04: +-05: SYSCALL (o64) +-06: CLTS +-07: SYSRET (o64) +-08: INVD +-09: WBINVD +-0a: +-0b: UD2 (1B) +-0c: +-# AMD's prefetch group. Intel supports prefetchw(/1) only. +-0d: GrpP +-0e: FEMMS +-# 3DNow! uses the last imm byte as opcode extension. +-0f: 3DNow! Pq,Qq,Ib +-# 0x0f 0x10-0x1f +-# NOTE: According to Intel SDM opcode map, vmovups and vmovupd has no operands +-# but it actually has operands. And also, vmovss and vmovsd only accept 128bit. +-# MOVSS/MOVSD has too many forms(3) on SDM. This map just shows a typical form. +-# Many AVX instructions lack v1 superscript, according to Intel AVX-Prgramming +-# Reference A.1 +-10: vmovups Vps,Wps | vmovupd Vpd,Wpd (66) | vmovss Vx,Hx,Wss (F3),(v1) | vmovsd Vx,Hx,Wsd (F2),(v1) +-11: vmovups Wps,Vps | vmovupd Wpd,Vpd (66) | vmovss Wss,Hx,Vss (F3),(v1) | vmovsd Wsd,Hx,Vsd (F2),(v1) +-12: vmovlps Vq,Hq,Mq (v1) | vmovhlps Vq,Hq,Uq (v1) | vmovlpd Vq,Hq,Mq (66),(v1) | vmovsldup Vx,Wx (F3) | vmovddup Vx,Wx (F2) +-13: vmovlps Mq,Vq (v1) | vmovlpd Mq,Vq (66),(v1) +-14: vunpcklps Vx,Hx,Wx | vunpcklpd Vx,Hx,Wx (66) +-15: vunpckhps Vx,Hx,Wx | vunpckhpd Vx,Hx,Wx (66) +-16: vmovhps Vdq,Hq,Mq (v1) | vmovlhps Vdq,Hq,Uq (v1) | vmovhpd Vdq,Hq,Mq (66),(v1) | vmovshdup Vx,Wx (F3) +-17: vmovhps Mq,Vq (v1) | vmovhpd Mq,Vq (66),(v1) +-18: Grp16 (1A) +-19: +-# Intel SDM opcode map does not list MPX instructions. For now using Gv for +-# bnd registers and Ev for everything else is OK because the instruction +-# decoder does not use the information except as an indication that there is +-# a ModR/M byte. +-1a: BNDCL Gv,Ev (F3) | BNDCU Gv,Ev (F2) | BNDMOV Gv,Ev (66) | BNDLDX Gv,Ev +-1b: BNDCN Gv,Ev (F2) | BNDMOV Ev,Gv (66) | BNDMK Gv,Ev (F3) | BNDSTX Ev,Gv +-1c: +-1d: +-1e: +-1f: NOP Ev +-# 0x0f 0x20-0x2f +-20: MOV Rd,Cd +-21: MOV Rd,Dd +-22: MOV Cd,Rd +-23: MOV Dd,Rd +-24: +-25: +-26: +-27: +-28: vmovaps Vps,Wps | vmovapd Vpd,Wpd (66) +-29: vmovaps Wps,Vps | vmovapd Wpd,Vpd (66) +-2a: cvtpi2ps Vps,Qpi | cvtpi2pd Vpd,Qpi (66) | vcvtsi2ss Vss,Hss,Ey (F3),(v1) | vcvtsi2sd Vsd,Hsd,Ey (F2),(v1) +-2b: vmovntps Mps,Vps | vmovntpd Mpd,Vpd (66) +-2c: cvttps2pi Ppi,Wps | cvttpd2pi Ppi,Wpd (66) | vcvttss2si Gy,Wss (F3),(v1) | vcvttsd2si Gy,Wsd (F2),(v1) +-2d: cvtps2pi Ppi,Wps | cvtpd2pi Qpi,Wpd (66) | vcvtss2si Gy,Wss (F3),(v1) | vcvtsd2si Gy,Wsd (F2),(v1) +-2e: vucomiss Vss,Wss (v1) | vucomisd Vsd,Wsd (66),(v1) +-2f: vcomiss Vss,Wss (v1) | vcomisd Vsd,Wsd (66),(v1) +-# 0x0f 0x30-0x3f +-30: WRMSR +-31: RDTSC +-32: RDMSR +-33: RDPMC +-34: SYSENTER +-35: SYSEXIT +-36: +-37: GETSEC +-38: escape # 3-byte escape 1 +-39: +-3a: escape # 3-byte escape 2 +-3b: +-3c: +-3d: +-3e: +-3f: +-# 0x0f 0x40-0x4f +-40: CMOVO Gv,Ev +-41: CMOVNO Gv,Ev | kandw/q Vk,Hk,Uk | kandb/d Vk,Hk,Uk (66) +-42: CMOVB/C/NAE Gv,Ev | kandnw/q Vk,Hk,Uk | kandnb/d Vk,Hk,Uk (66) +-43: CMOVAE/NB/NC Gv,Ev +-44: CMOVE/Z Gv,Ev | knotw/q Vk,Uk | knotb/d Vk,Uk (66) +-45: CMOVNE/NZ Gv,Ev | korw/q Vk,Hk,Uk | korb/d Vk,Hk,Uk (66) +-46: CMOVBE/NA Gv,Ev | kxnorw/q Vk,Hk,Uk | kxnorb/d Vk,Hk,Uk (66) +-47: CMOVA/NBE Gv,Ev | kxorw/q Vk,Hk,Uk | kxorb/d Vk,Hk,Uk (66) +-48: CMOVS Gv,Ev +-49: CMOVNS Gv,Ev +-4a: CMOVP/PE Gv,Ev | kaddw/q Vk,Hk,Uk | kaddb/d Vk,Hk,Uk (66) +-4b: CMOVNP/PO Gv,Ev | kunpckbw Vk,Hk,Uk (66) | kunpckwd/dq Vk,Hk,Uk +-4c: CMOVL/NGE Gv,Ev +-4d: CMOVNL/GE Gv,Ev +-4e: CMOVLE/NG Gv,Ev +-4f: CMOVNLE/G Gv,Ev +-# 0x0f 0x50-0x5f +-50: vmovmskps Gy,Ups | vmovmskpd Gy,Upd (66) +-51: vsqrtps Vps,Wps | vsqrtpd Vpd,Wpd (66) | vsqrtss Vss,Hss,Wss (F3),(v1) | vsqrtsd Vsd,Hsd,Wsd (F2),(v1) +-52: vrsqrtps Vps,Wps | vrsqrtss Vss,Hss,Wss (F3),(v1) +-53: vrcpps Vps,Wps | vrcpss Vss,Hss,Wss (F3),(v1) +-54: vandps Vps,Hps,Wps | vandpd Vpd,Hpd,Wpd (66) +-55: vandnps Vps,Hps,Wps | vandnpd Vpd,Hpd,Wpd (66) +-56: vorps Vps,Hps,Wps | vorpd Vpd,Hpd,Wpd (66) +-57: vxorps Vps,Hps,Wps | vxorpd Vpd,Hpd,Wpd (66) +-58: vaddps Vps,Hps,Wps | vaddpd Vpd,Hpd,Wpd (66) | vaddss Vss,Hss,Wss (F3),(v1) | vaddsd Vsd,Hsd,Wsd (F2),(v1) +-59: vmulps Vps,Hps,Wps | vmulpd Vpd,Hpd,Wpd (66) | vmulss Vss,Hss,Wss (F3),(v1) | vmulsd Vsd,Hsd,Wsd (F2),(v1) +-5a: vcvtps2pd Vpd,Wps | vcvtpd2ps Vps,Wpd (66) | vcvtss2sd Vsd,Hx,Wss (F3),(v1) | vcvtsd2ss Vss,Hx,Wsd (F2),(v1) +-5b: vcvtdq2ps Vps,Wdq | vcvtqq2ps Vps,Wqq (evo) | vcvtps2dq Vdq,Wps (66) | vcvttps2dq Vdq,Wps (F3) +-5c: vsubps Vps,Hps,Wps | vsubpd Vpd,Hpd,Wpd (66) | vsubss Vss,Hss,Wss (F3),(v1) | vsubsd Vsd,Hsd,Wsd (F2),(v1) +-5d: vminps Vps,Hps,Wps | vminpd Vpd,Hpd,Wpd (66) | vminss Vss,Hss,Wss (F3),(v1) | vminsd Vsd,Hsd,Wsd (F2),(v1) +-5e: vdivps Vps,Hps,Wps | vdivpd Vpd,Hpd,Wpd (66) | vdivss Vss,Hss,Wss (F3),(v1) | vdivsd Vsd,Hsd,Wsd (F2),(v1) +-5f: vmaxps Vps,Hps,Wps | vmaxpd Vpd,Hpd,Wpd (66) | vmaxss Vss,Hss,Wss (F3),(v1) | vmaxsd Vsd,Hsd,Wsd (F2),(v1) +-# 0x0f 0x60-0x6f +-60: punpcklbw Pq,Qd | vpunpcklbw Vx,Hx,Wx (66),(v1) +-61: punpcklwd Pq,Qd | vpunpcklwd Vx,Hx,Wx (66),(v1) +-62: punpckldq Pq,Qd | vpunpckldq Vx,Hx,Wx (66),(v1) +-63: packsswb Pq,Qq | vpacksswb Vx,Hx,Wx (66),(v1) +-64: pcmpgtb Pq,Qq | vpcmpgtb Vx,Hx,Wx (66),(v1) +-65: pcmpgtw Pq,Qq | vpcmpgtw Vx,Hx,Wx (66),(v1) +-66: pcmpgtd Pq,Qq | vpcmpgtd Vx,Hx,Wx (66),(v1) +-67: packuswb Pq,Qq | vpackuswb Vx,Hx,Wx (66),(v1) +-68: punpckhbw Pq,Qd | vpunpckhbw Vx,Hx,Wx (66),(v1) +-69: punpckhwd Pq,Qd | vpunpckhwd Vx,Hx,Wx (66),(v1) +-6a: punpckhdq Pq,Qd | vpunpckhdq Vx,Hx,Wx (66),(v1) +-6b: packssdw Pq,Qd | vpackssdw Vx,Hx,Wx (66),(v1) +-6c: vpunpcklqdq Vx,Hx,Wx (66),(v1) +-6d: vpunpckhqdq Vx,Hx,Wx (66),(v1) +-6e: movd/q Pd,Ey | vmovd/q Vy,Ey (66),(v1) +-6f: movq Pq,Qq | vmovdqa Vx,Wx (66) | vmovdqa32/64 Vx,Wx (66),(evo) | vmovdqu Vx,Wx (F3) | vmovdqu32/64 Vx,Wx (F3),(evo) | vmovdqu8/16 Vx,Wx (F2),(ev) +-# 0x0f 0x70-0x7f +-70: pshufw Pq,Qq,Ib | vpshufd Vx,Wx,Ib (66),(v1) | vpshufhw Vx,Wx,Ib (F3),(v1) | vpshuflw Vx,Wx,Ib (F2),(v1) +-71: Grp12 (1A) +-72: Grp13 (1A) +-73: Grp14 (1A) +-74: pcmpeqb Pq,Qq | vpcmpeqb Vx,Hx,Wx (66),(v1) +-75: pcmpeqw Pq,Qq | vpcmpeqw Vx,Hx,Wx (66),(v1) +-76: pcmpeqd Pq,Qq | vpcmpeqd Vx,Hx,Wx (66),(v1) +-# Note: Remove (v), because vzeroall and vzeroupper becomes emms without VEX. +-77: emms | vzeroupper | vzeroall +-78: VMREAD Ey,Gy | vcvttps2udq/pd2udq Vx,Wpd (evo) | vcvttsd2usi Gv,Wx (F2),(ev) | vcvttss2usi Gv,Wx (F3),(ev) | vcvttps2uqq/pd2uqq Vx,Wx (66),(ev) +-79: VMWRITE Gy,Ey | vcvtps2udq/pd2udq Vx,Wpd (evo) | vcvtsd2usi Gv,Wx (F2),(ev) | vcvtss2usi Gv,Wx (F3),(ev) | vcvtps2uqq/pd2uqq Vx,Wx (66),(ev) +-7a: vcvtudq2pd/uqq2pd Vpd,Wx (F3),(ev) | vcvtudq2ps/uqq2ps Vpd,Wx (F2),(ev) | vcvttps2qq/pd2qq Vx,Wx (66),(ev) +-7b: vcvtusi2sd Vpd,Hpd,Ev (F2),(ev) | vcvtusi2ss Vps,Hps,Ev (F3),(ev) | vcvtps2qq/pd2qq Vx,Wx (66),(ev) +-7c: vhaddpd Vpd,Hpd,Wpd (66) | vhaddps Vps,Hps,Wps (F2) +-7d: vhsubpd Vpd,Hpd,Wpd (66) | vhsubps Vps,Hps,Wps (F2) +-7e: movd/q Ey,Pd | vmovd/q Ey,Vy (66),(v1) | vmovq Vq,Wq (F3),(v1) +-7f: movq Qq,Pq | vmovdqa Wx,Vx (66) | vmovdqa32/64 Wx,Vx (66),(evo) | vmovdqu Wx,Vx (F3) | vmovdqu32/64 Wx,Vx (F3),(evo) | vmovdqu8/16 Wx,Vx (F2),(ev) +-# 0x0f 0x80-0x8f +-# Note: "forced64" is Intel CPU behavior (see comment about CALL insn). +-80: JO Jz (f64) +-81: JNO Jz (f64) +-82: JB/JC/JNAE Jz (f64) +-83: JAE/JNB/JNC Jz (f64) +-84: JE/JZ Jz (f64) +-85: JNE/JNZ Jz (f64) +-86: JBE/JNA Jz (f64) +-87: JA/JNBE Jz (f64) +-88: JS Jz (f64) +-89: JNS Jz (f64) +-8a: JP/JPE Jz (f64) +-8b: JNP/JPO Jz (f64) +-8c: JL/JNGE Jz (f64) +-8d: JNL/JGE Jz (f64) +-8e: JLE/JNG Jz (f64) +-8f: JNLE/JG Jz (f64) +-# 0x0f 0x90-0x9f +-90: SETO Eb | kmovw/q Vk,Wk | kmovb/d Vk,Wk (66) +-91: SETNO Eb | kmovw/q Mv,Vk | kmovb/d Mv,Vk (66) +-92: SETB/C/NAE Eb | kmovw Vk,Rv | kmovb Vk,Rv (66) | kmovq/d Vk,Rv (F2) +-93: SETAE/NB/NC Eb | kmovw Gv,Uk | kmovb Gv,Uk (66) | kmovq/d Gv,Uk (F2) +-94: SETE/Z Eb +-95: SETNE/NZ Eb +-96: SETBE/NA Eb +-97: SETA/NBE Eb +-98: SETS Eb | kortestw/q Vk,Uk | kortestb/d Vk,Uk (66) +-99: SETNS Eb | ktestw/q Vk,Uk | ktestb/d Vk,Uk (66) +-9a: SETP/PE Eb +-9b: SETNP/PO Eb +-9c: SETL/NGE Eb +-9d: SETNL/GE Eb +-9e: SETLE/NG Eb +-9f: SETNLE/G Eb +-# 0x0f 0xa0-0xaf +-a0: PUSH FS (d64) +-a1: POP FS (d64) +-a2: CPUID +-a3: BT Ev,Gv +-a4: SHLD Ev,Gv,Ib +-a5: SHLD Ev,Gv,CL +-a6: GrpPDLK +-a7: GrpRNG +-a8: PUSH GS (d64) +-a9: POP GS (d64) +-aa: RSM +-ab: BTS Ev,Gv +-ac: SHRD Ev,Gv,Ib +-ad: SHRD Ev,Gv,CL +-ae: Grp15 (1A),(1C) +-af: IMUL Gv,Ev +-# 0x0f 0xb0-0xbf +-b0: CMPXCHG Eb,Gb +-b1: CMPXCHG Ev,Gv +-b2: LSS Gv,Mp +-b3: BTR Ev,Gv +-b4: LFS Gv,Mp +-b5: LGS Gv,Mp +-b6: MOVZX Gv,Eb +-b7: MOVZX Gv,Ew +-b8: JMPE (!F3) | POPCNT Gv,Ev (F3) +-b9: Grp10 (1A) +-ba: Grp8 Ev,Ib (1A) +-bb: BTC Ev,Gv +-bc: BSF Gv,Ev (!F3) | TZCNT Gv,Ev (F3) +-bd: BSR Gv,Ev (!F3) | LZCNT Gv,Ev (F3) +-be: MOVSX Gv,Eb +-bf: MOVSX Gv,Ew +-# 0x0f 0xc0-0xcf +-c0: XADD Eb,Gb +-c1: XADD Ev,Gv +-c2: vcmpps Vps,Hps,Wps,Ib | vcmppd Vpd,Hpd,Wpd,Ib (66) | vcmpss Vss,Hss,Wss,Ib (F3),(v1) | vcmpsd Vsd,Hsd,Wsd,Ib (F2),(v1) +-c3: movnti My,Gy +-c4: pinsrw Pq,Ry/Mw,Ib | vpinsrw Vdq,Hdq,Ry/Mw,Ib (66),(v1) +-c5: pextrw Gd,Nq,Ib | vpextrw Gd,Udq,Ib (66),(v1) +-c6: vshufps Vps,Hps,Wps,Ib | vshufpd Vpd,Hpd,Wpd,Ib (66) +-c7: Grp9 (1A) +-c8: BSWAP RAX/EAX/R8/R8D +-c9: BSWAP RCX/ECX/R9/R9D +-ca: BSWAP RDX/EDX/R10/R10D +-cb: BSWAP RBX/EBX/R11/R11D +-cc: BSWAP RSP/ESP/R12/R12D +-cd: BSWAP RBP/EBP/R13/R13D +-ce: BSWAP RSI/ESI/R14/R14D +-cf: BSWAP RDI/EDI/R15/R15D +-# 0x0f 0xd0-0xdf +-d0: vaddsubpd Vpd,Hpd,Wpd (66) | vaddsubps Vps,Hps,Wps (F2) +-d1: psrlw Pq,Qq | vpsrlw Vx,Hx,Wx (66),(v1) +-d2: psrld Pq,Qq | vpsrld Vx,Hx,Wx (66),(v1) +-d3: psrlq Pq,Qq | vpsrlq Vx,Hx,Wx (66),(v1) +-d4: paddq Pq,Qq | vpaddq Vx,Hx,Wx (66),(v1) +-d5: pmullw Pq,Qq | vpmullw Vx,Hx,Wx (66),(v1) +-d6: vmovq Wq,Vq (66),(v1) | movq2dq Vdq,Nq (F3) | movdq2q Pq,Uq (F2) +-d7: pmovmskb Gd,Nq | vpmovmskb Gd,Ux (66),(v1) +-d8: psubusb Pq,Qq | vpsubusb Vx,Hx,Wx (66),(v1) +-d9: psubusw Pq,Qq | vpsubusw Vx,Hx,Wx (66),(v1) +-da: pminub Pq,Qq | vpminub Vx,Hx,Wx (66),(v1) +-db: pand Pq,Qq | vpand Vx,Hx,Wx (66),(v1) | vpandd/q Vx,Hx,Wx (66),(evo) +-dc: paddusb Pq,Qq | vpaddusb Vx,Hx,Wx (66),(v1) +-dd: paddusw Pq,Qq | vpaddusw Vx,Hx,Wx (66),(v1) +-de: pmaxub Pq,Qq | vpmaxub Vx,Hx,Wx (66),(v1) +-df: pandn Pq,Qq | vpandn Vx,Hx,Wx (66),(v1) | vpandnd/q Vx,Hx,Wx (66),(evo) +-# 0x0f 0xe0-0xef +-e0: pavgb Pq,Qq | vpavgb Vx,Hx,Wx (66),(v1) +-e1: psraw Pq,Qq | vpsraw Vx,Hx,Wx (66),(v1) +-e2: psrad Pq,Qq | vpsrad Vx,Hx,Wx (66),(v1) +-e3: pavgw Pq,Qq | vpavgw Vx,Hx,Wx (66),(v1) +-e4: pmulhuw Pq,Qq | vpmulhuw Vx,Hx,Wx (66),(v1) +-e5: pmulhw Pq,Qq | vpmulhw Vx,Hx,Wx (66),(v1) +-e6: vcvttpd2dq Vx,Wpd (66) | vcvtdq2pd Vx,Wdq (F3) | vcvtdq2pd/qq2pd Vx,Wdq (F3),(evo) | vcvtpd2dq Vx,Wpd (F2) +-e7: movntq Mq,Pq | vmovntdq Mx,Vx (66) +-e8: psubsb Pq,Qq | vpsubsb Vx,Hx,Wx (66),(v1) +-e9: psubsw Pq,Qq | vpsubsw Vx,Hx,Wx (66),(v1) +-ea: pminsw Pq,Qq | vpminsw Vx,Hx,Wx (66),(v1) +-eb: por Pq,Qq | vpor Vx,Hx,Wx (66),(v1) | vpord/q Vx,Hx,Wx (66),(evo) +-ec: paddsb Pq,Qq | vpaddsb Vx,Hx,Wx (66),(v1) +-ed: paddsw Pq,Qq | vpaddsw Vx,Hx,Wx (66),(v1) +-ee: pmaxsw Pq,Qq | vpmaxsw Vx,Hx,Wx (66),(v1) +-ef: pxor Pq,Qq | vpxor Vx,Hx,Wx (66),(v1) | vpxord/q Vx,Hx,Wx (66),(evo) +-# 0x0f 0xf0-0xff +-f0: vlddqu Vx,Mx (F2) +-f1: psllw Pq,Qq | vpsllw Vx,Hx,Wx (66),(v1) +-f2: pslld Pq,Qq | vpslld Vx,Hx,Wx (66),(v1) +-f3: psllq Pq,Qq | vpsllq Vx,Hx,Wx (66),(v1) +-f4: pmuludq Pq,Qq | vpmuludq Vx,Hx,Wx (66),(v1) +-f5: pmaddwd Pq,Qq | vpmaddwd Vx,Hx,Wx (66),(v1) +-f6: psadbw Pq,Qq | vpsadbw Vx,Hx,Wx (66),(v1) +-f7: maskmovq Pq,Nq | vmaskmovdqu Vx,Ux (66),(v1) +-f8: psubb Pq,Qq | vpsubb Vx,Hx,Wx (66),(v1) +-f9: psubw Pq,Qq | vpsubw Vx,Hx,Wx (66),(v1) +-fa: psubd Pq,Qq | vpsubd Vx,Hx,Wx (66),(v1) +-fb: psubq Pq,Qq | vpsubq Vx,Hx,Wx (66),(v1) +-fc: paddb Pq,Qq | vpaddb Vx,Hx,Wx (66),(v1) +-fd: paddw Pq,Qq | vpaddw Vx,Hx,Wx (66),(v1) +-fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1) +-ff: +-EndTable +- +-Table: 3-byte opcode 1 (0x0f 0x38) +-Referrer: 3-byte escape 1 +-AVXcode: 2 +-# 0x0f 0x38 0x00-0x0f +-00: pshufb Pq,Qq | vpshufb Vx,Hx,Wx (66),(v1) +-01: phaddw Pq,Qq | vphaddw Vx,Hx,Wx (66),(v1) +-02: phaddd Pq,Qq | vphaddd Vx,Hx,Wx (66),(v1) +-03: phaddsw Pq,Qq | vphaddsw Vx,Hx,Wx (66),(v1) +-04: pmaddubsw Pq,Qq | vpmaddubsw Vx,Hx,Wx (66),(v1) +-05: phsubw Pq,Qq | vphsubw Vx,Hx,Wx (66),(v1) +-06: phsubd Pq,Qq | vphsubd Vx,Hx,Wx (66),(v1) +-07: phsubsw Pq,Qq | vphsubsw Vx,Hx,Wx (66),(v1) +-08: psignb Pq,Qq | vpsignb Vx,Hx,Wx (66),(v1) +-09: psignw Pq,Qq | vpsignw Vx,Hx,Wx (66),(v1) +-0a: psignd Pq,Qq | vpsignd Vx,Hx,Wx (66),(v1) +-0b: pmulhrsw Pq,Qq | vpmulhrsw Vx,Hx,Wx (66),(v1) +-0c: vpermilps Vx,Hx,Wx (66),(v) +-0d: vpermilpd Vx,Hx,Wx (66),(v) +-0e: vtestps Vx,Wx (66),(v) +-0f: vtestpd Vx,Wx (66),(v) +-# 0x0f 0x38 0x10-0x1f +-10: pblendvb Vdq,Wdq (66) | vpsrlvw Vx,Hx,Wx (66),(evo) | vpmovuswb Wx,Vx (F3),(ev) +-11: vpmovusdb Wx,Vd (F3),(ev) | vpsravw Vx,Hx,Wx (66),(ev) +-12: vpmovusqb Wx,Vq (F3),(ev) | vpsllvw Vx,Hx,Wx (66),(ev) +-13: vcvtph2ps Vx,Wx (66),(v) | vpmovusdw Wx,Vd (F3),(ev) +-14: blendvps Vdq,Wdq (66) | vpmovusqw Wx,Vq (F3),(ev) | vprorvd/q Vx,Hx,Wx (66),(evo) +-15: blendvpd Vdq,Wdq (66) | vpmovusqd Wx,Vq (F3),(ev) | vprolvd/q Vx,Hx,Wx (66),(evo) +-16: vpermps Vqq,Hqq,Wqq (66),(v) | vpermps/d Vqq,Hqq,Wqq (66),(evo) +-17: vptest Vx,Wx (66) +-18: vbroadcastss Vx,Wd (66),(v) +-19: vbroadcastsd Vqq,Wq (66),(v) | vbroadcastf32x2 Vqq,Wq (66),(evo) +-1a: vbroadcastf128 Vqq,Mdq (66),(v) | vbroadcastf32x4/64x2 Vqq,Wq (66),(evo) +-1b: vbroadcastf32x8/64x4 Vqq,Mdq (66),(ev) +-1c: pabsb Pq,Qq | vpabsb Vx,Wx (66),(v1) +-1d: pabsw Pq,Qq | vpabsw Vx,Wx (66),(v1) +-1e: pabsd Pq,Qq | vpabsd Vx,Wx (66),(v1) +-1f: vpabsq Vx,Wx (66),(ev) +-# 0x0f 0x38 0x20-0x2f +-20: vpmovsxbw Vx,Ux/Mq (66),(v1) | vpmovswb Wx,Vx (F3),(ev) +-21: vpmovsxbd Vx,Ux/Md (66),(v1) | vpmovsdb Wx,Vd (F3),(ev) +-22: vpmovsxbq Vx,Ux/Mw (66),(v1) | vpmovsqb Wx,Vq (F3),(ev) +-23: vpmovsxwd Vx,Ux/Mq (66),(v1) | vpmovsdw Wx,Vd (F3),(ev) +-24: vpmovsxwq Vx,Ux/Md (66),(v1) | vpmovsqw Wx,Vq (F3),(ev) +-25: vpmovsxdq Vx,Ux/Mq (66),(v1) | vpmovsqd Wx,Vq (F3),(ev) +-26: vptestmb/w Vk,Hx,Wx (66),(ev) | vptestnmb/w Vk,Hx,Wx (F3),(ev) +-27: vptestmd/q Vk,Hx,Wx (66),(ev) | vptestnmd/q Vk,Hx,Wx (F3),(ev) +-28: vpmuldq Vx,Hx,Wx (66),(v1) | vpmovm2b/w Vx,Uk (F3),(ev) +-29: vpcmpeqq Vx,Hx,Wx (66),(v1) | vpmovb2m/w2m Vk,Ux (F3),(ev) +-2a: vmovntdqa Vx,Mx (66),(v1) | vpbroadcastmb2q Vx,Uk (F3),(ev) +-2b: vpackusdw Vx,Hx,Wx (66),(v1) +-2c: vmaskmovps Vx,Hx,Mx (66),(v) | vscalefps/d Vx,Hx,Wx (66),(evo) +-2d: vmaskmovpd Vx,Hx,Mx (66),(v) | vscalefss/d Vx,Hx,Wx (66),(evo) +-2e: vmaskmovps Mx,Hx,Vx (66),(v) +-2f: vmaskmovpd Mx,Hx,Vx (66),(v) +-# 0x0f 0x38 0x30-0x3f +-30: vpmovzxbw Vx,Ux/Mq (66),(v1) | vpmovwb Wx,Vx (F3),(ev) +-31: vpmovzxbd Vx,Ux/Md (66),(v1) | vpmovdb Wx,Vd (F3),(ev) +-32: vpmovzxbq Vx,Ux/Mw (66),(v1) | vpmovqb Wx,Vq (F3),(ev) +-33: vpmovzxwd Vx,Ux/Mq (66),(v1) | vpmovdw Wx,Vd (F3),(ev) +-34: vpmovzxwq Vx,Ux/Md (66),(v1) | vpmovqw Wx,Vq (F3),(ev) +-35: vpmovzxdq Vx,Ux/Mq (66),(v1) | vpmovqd Wx,Vq (F3),(ev) +-36: vpermd Vqq,Hqq,Wqq (66),(v) | vpermd/q Vqq,Hqq,Wqq (66),(evo) +-37: vpcmpgtq Vx,Hx,Wx (66),(v1) +-38: vpminsb Vx,Hx,Wx (66),(v1) | vpmovm2d/q Vx,Uk (F3),(ev) +-39: vpminsd Vx,Hx,Wx (66),(v1) | vpminsd/q Vx,Hx,Wx (66),(evo) | vpmovd2m/q2m Vk,Ux (F3),(ev) +-3a: vpminuw Vx,Hx,Wx (66),(v1) | vpbroadcastmw2d Vx,Uk (F3),(ev) +-3b: vpminud Vx,Hx,Wx (66),(v1) | vpminud/q Vx,Hx,Wx (66),(evo) +-3c: vpmaxsb Vx,Hx,Wx (66),(v1) +-3d: vpmaxsd Vx,Hx,Wx (66),(v1) | vpmaxsd/q Vx,Hx,Wx (66),(evo) +-3e: vpmaxuw Vx,Hx,Wx (66),(v1) +-3f: vpmaxud Vx,Hx,Wx (66),(v1) | vpmaxud/q Vx,Hx,Wx (66),(evo) +-# 0x0f 0x38 0x40-0x8f +-40: vpmulld Vx,Hx,Wx (66),(v1) | vpmulld/q Vx,Hx,Wx (66),(evo) +-41: vphminposuw Vdq,Wdq (66),(v1) +-42: vgetexpps/d Vx,Wx (66),(ev) +-43: vgetexpss/d Vx,Hx,Wx (66),(ev) +-44: vplzcntd/q Vx,Wx (66),(ev) +-45: vpsrlvd/q Vx,Hx,Wx (66),(v) +-46: vpsravd Vx,Hx,Wx (66),(v) | vpsravd/q Vx,Hx,Wx (66),(evo) +-47: vpsllvd/q Vx,Hx,Wx (66),(v) +-# Skip 0x48-0x4b +-4c: vrcp14ps/d Vpd,Wpd (66),(ev) +-4d: vrcp14ss/d Vsd,Hpd,Wsd (66),(ev) +-4e: vrsqrt14ps/d Vpd,Wpd (66),(ev) +-4f: vrsqrt14ss/d Vsd,Hsd,Wsd (66),(ev) +-# Skip 0x50-0x57 +-58: vpbroadcastd Vx,Wx (66),(v) +-59: vpbroadcastq Vx,Wx (66),(v) | vbroadcasti32x2 Vx,Wx (66),(evo) +-5a: vbroadcasti128 Vqq,Mdq (66),(v) | vbroadcasti32x4/64x2 Vx,Wx (66),(evo) +-5b: vbroadcasti32x8/64x4 Vqq,Mdq (66),(ev) +-# Skip 0x5c-0x63 +-64: vpblendmd/q Vx,Hx,Wx (66),(ev) +-65: vblendmps/d Vx,Hx,Wx (66),(ev) +-66: vpblendmb/w Vx,Hx,Wx (66),(ev) +-# Skip 0x67-0x74 +-75: vpermi2b/w Vx,Hx,Wx (66),(ev) +-76: vpermi2d/q Vx,Hx,Wx (66),(ev) +-77: vpermi2ps/d Vx,Hx,Wx (66),(ev) +-78: vpbroadcastb Vx,Wx (66),(v) +-79: vpbroadcastw Vx,Wx (66),(v) +-7a: vpbroadcastb Vx,Rv (66),(ev) +-7b: vpbroadcastw Vx,Rv (66),(ev) +-7c: vpbroadcastd/q Vx,Rv (66),(ev) +-7d: vpermt2b/w Vx,Hx,Wx (66),(ev) +-7e: vpermt2d/q Vx,Hx,Wx (66),(ev) +-7f: vpermt2ps/d Vx,Hx,Wx (66),(ev) +-80: INVEPT Gy,Mdq (66) +-81: INVPID Gy,Mdq (66) +-82: INVPCID Gy,Mdq (66) +-83: vpmultishiftqb Vx,Hx,Wx (66),(ev) +-88: vexpandps/d Vpd,Wpd (66),(ev) +-89: vpexpandd/q Vx,Wx (66),(ev) +-8a: vcompressps/d Wx,Vx (66),(ev) +-8b: vpcompressd/q Wx,Vx (66),(ev) +-8c: vpmaskmovd/q Vx,Hx,Mx (66),(v) +-8d: vpermb/w Vx,Hx,Wx (66),(ev) +-8e: vpmaskmovd/q Mx,Vx,Hx (66),(v) +-# 0x0f 0x38 0x90-0xbf (FMA) +-90: vgatherdd/q Vx,Hx,Wx (66),(v) | vpgatherdd/q Vx,Wx (66),(evo) +-91: vgatherqd/q Vx,Hx,Wx (66),(v) | vpgatherqd/q Vx,Wx (66),(evo) +-92: vgatherdps/d Vx,Hx,Wx (66),(v) +-93: vgatherqps/d Vx,Hx,Wx (66),(v) +-94: +-95: +-96: vfmaddsub132ps/d Vx,Hx,Wx (66),(v) +-97: vfmsubadd132ps/d Vx,Hx,Wx (66),(v) +-98: vfmadd132ps/d Vx,Hx,Wx (66),(v) +-99: vfmadd132ss/d Vx,Hx,Wx (66),(v),(v1) +-9a: vfmsub132ps/d Vx,Hx,Wx (66),(v) +-9b: vfmsub132ss/d Vx,Hx,Wx (66),(v),(v1) +-9c: vfnmadd132ps/d Vx,Hx,Wx (66),(v) +-9d: vfnmadd132ss/d Vx,Hx,Wx (66),(v),(v1) +-9e: vfnmsub132ps/d Vx,Hx,Wx (66),(v) +-9f: vfnmsub132ss/d Vx,Hx,Wx (66),(v),(v1) +-a0: vpscatterdd/q Wx,Vx (66),(ev) +-a1: vpscatterqd/q Wx,Vx (66),(ev) +-a2: vscatterdps/d Wx,Vx (66),(ev) +-a3: vscatterqps/d Wx,Vx (66),(ev) +-a6: vfmaddsub213ps/d Vx,Hx,Wx (66),(v) +-a7: vfmsubadd213ps/d Vx,Hx,Wx (66),(v) +-a8: vfmadd213ps/d Vx,Hx,Wx (66),(v) +-a9: vfmadd213ss/d Vx,Hx,Wx (66),(v),(v1) +-aa: vfmsub213ps/d Vx,Hx,Wx (66),(v) +-ab: vfmsub213ss/d Vx,Hx,Wx (66),(v),(v1) +-ac: vfnmadd213ps/d Vx,Hx,Wx (66),(v) +-ad: vfnmadd213ss/d Vx,Hx,Wx (66),(v),(v1) +-ae: vfnmsub213ps/d Vx,Hx,Wx (66),(v) +-af: vfnmsub213ss/d Vx,Hx,Wx (66),(v),(v1) +-b4: vpmadd52luq Vx,Hx,Wx (66),(ev) +-b5: vpmadd52huq Vx,Hx,Wx (66),(ev) +-b6: vfmaddsub231ps/d Vx,Hx,Wx (66),(v) +-b7: vfmsubadd231ps/d Vx,Hx,Wx (66),(v) +-b8: vfmadd231ps/d Vx,Hx,Wx (66),(v) +-b9: vfmadd231ss/d Vx,Hx,Wx (66),(v),(v1) +-ba: vfmsub231ps/d Vx,Hx,Wx (66),(v) +-bb: vfmsub231ss/d Vx,Hx,Wx (66),(v),(v1) +-bc: vfnmadd231ps/d Vx,Hx,Wx (66),(v) +-bd: vfnmadd231ss/d Vx,Hx,Wx (66),(v),(v1) +-be: vfnmsub231ps/d Vx,Hx,Wx (66),(v) +-bf: vfnmsub231ss/d Vx,Hx,Wx (66),(v),(v1) +-# 0x0f 0x38 0xc0-0xff +-c4: vpconflictd/q Vx,Wx (66),(ev) +-c6: Grp18 (1A) +-c7: Grp19 (1A) +-c8: sha1nexte Vdq,Wdq | vexp2ps/d Vx,Wx (66),(ev) +-c9: sha1msg1 Vdq,Wdq +-ca: sha1msg2 Vdq,Wdq | vrcp28ps/d Vx,Wx (66),(ev) +-cb: sha256rnds2 Vdq,Wdq | vrcp28ss/d Vx,Hx,Wx (66),(ev) +-cc: sha256msg1 Vdq,Wdq | vrsqrt28ps/d Vx,Wx (66),(ev) +-cd: sha256msg2 Vdq,Wdq | vrsqrt28ss/d Vx,Hx,Wx (66),(ev) +-db: VAESIMC Vdq,Wdq (66),(v1) +-dc: VAESENC Vdq,Hdq,Wdq (66),(v1) +-dd: VAESENCLAST Vdq,Hdq,Wdq (66),(v1) +-de: VAESDEC Vdq,Hdq,Wdq (66),(v1) +-df: VAESDECLAST Vdq,Hdq,Wdq (66),(v1) +-f0: MOVBE Gy,My | MOVBE Gw,Mw (66) | CRC32 Gd,Eb (F2) | CRC32 Gd,Eb (66&F2) +-f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2) +-f2: ANDN Gy,By,Ey (v) +-f3: Grp17 (1A) +-f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) +-f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) +-f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v) +-EndTable +- +-Table: 3-byte opcode 2 (0x0f 0x3a) +-Referrer: 3-byte escape 2 +-AVXcode: 3 +-# 0x0f 0x3a 0x00-0xff +-00: vpermq Vqq,Wqq,Ib (66),(v) +-01: vpermpd Vqq,Wqq,Ib (66),(v) +-02: vpblendd Vx,Hx,Wx,Ib (66),(v) +-03: valignd/q Vx,Hx,Wx,Ib (66),(ev) +-04: vpermilps Vx,Wx,Ib (66),(v) +-05: vpermilpd Vx,Wx,Ib (66),(v) +-06: vperm2f128 Vqq,Hqq,Wqq,Ib (66),(v) +-07: +-08: vroundps Vx,Wx,Ib (66) | vrndscaleps Vx,Wx,Ib (66),(evo) +-09: vroundpd Vx,Wx,Ib (66) | vrndscalepd Vx,Wx,Ib (66),(evo) +-0a: vroundss Vss,Wss,Ib (66),(v1) | vrndscaless Vx,Hx,Wx,Ib (66),(evo) +-0b: vroundsd Vsd,Wsd,Ib (66),(v1) | vrndscalesd Vx,Hx,Wx,Ib (66),(evo) +-0c: vblendps Vx,Hx,Wx,Ib (66) +-0d: vblendpd Vx,Hx,Wx,Ib (66) +-0e: vpblendw Vx,Hx,Wx,Ib (66),(v1) +-0f: palignr Pq,Qq,Ib | vpalignr Vx,Hx,Wx,Ib (66),(v1) +-14: vpextrb Rd/Mb,Vdq,Ib (66),(v1) +-15: vpextrw Rd/Mw,Vdq,Ib (66),(v1) +-16: vpextrd/q Ey,Vdq,Ib (66),(v1) +-17: vextractps Ed,Vdq,Ib (66),(v1) +-18: vinsertf128 Vqq,Hqq,Wqq,Ib (66),(v) | vinsertf32x4/64x2 Vqq,Hqq,Wqq,Ib (66),(evo) +-19: vextractf128 Wdq,Vqq,Ib (66),(v) | vextractf32x4/64x2 Wdq,Vqq,Ib (66),(evo) +-1a: vinsertf32x8/64x4 Vqq,Hqq,Wqq,Ib (66),(ev) +-1b: vextractf32x8/64x4 Wdq,Vqq,Ib (66),(ev) +-1d: vcvtps2ph Wx,Vx,Ib (66),(v) +-1e: vpcmpud/q Vk,Hd,Wd,Ib (66),(ev) +-1f: vpcmpd/q Vk,Hd,Wd,Ib (66),(ev) +-20: vpinsrb Vdq,Hdq,Ry/Mb,Ib (66),(v1) +-21: vinsertps Vdq,Hdq,Udq/Md,Ib (66),(v1) +-22: vpinsrd/q Vdq,Hdq,Ey,Ib (66),(v1) +-23: vshuff32x4/64x2 Vx,Hx,Wx,Ib (66),(ev) +-25: vpternlogd/q Vx,Hx,Wx,Ib (66),(ev) +-26: vgetmantps/d Vx,Wx,Ib (66),(ev) +-27: vgetmantss/d Vx,Hx,Wx,Ib (66),(ev) +-30: kshiftrb/w Vk,Uk,Ib (66),(v) +-31: kshiftrd/q Vk,Uk,Ib (66),(v) +-32: kshiftlb/w Vk,Uk,Ib (66),(v) +-33: kshiftld/q Vk,Uk,Ib (66),(v) +-38: vinserti128 Vqq,Hqq,Wqq,Ib (66),(v) | vinserti32x4/64x2 Vqq,Hqq,Wqq,Ib (66),(evo) +-39: vextracti128 Wdq,Vqq,Ib (66),(v) | vextracti32x4/64x2 Wdq,Vqq,Ib (66),(evo) +-3a: vinserti32x8/64x4 Vqq,Hqq,Wqq,Ib (66),(ev) +-3b: vextracti32x8/64x4 Wdq,Vqq,Ib (66),(ev) +-3e: vpcmpub/w Vk,Hk,Wx,Ib (66),(ev) +-3f: vpcmpb/w Vk,Hk,Wx,Ib (66),(ev) +-40: vdpps Vx,Hx,Wx,Ib (66) +-41: vdppd Vdq,Hdq,Wdq,Ib (66),(v1) +-42: vmpsadbw Vx,Hx,Wx,Ib (66),(v1) | vdbpsadbw Vx,Hx,Wx,Ib (66),(evo) +-43: vshufi32x4/64x2 Vx,Hx,Wx,Ib (66),(ev) +-44: vpclmulqdq Vdq,Hdq,Wdq,Ib (66),(v1) +-46: vperm2i128 Vqq,Hqq,Wqq,Ib (66),(v) +-4a: vblendvps Vx,Hx,Wx,Lx (66),(v) +-4b: vblendvpd Vx,Hx,Wx,Lx (66),(v) +-4c: vpblendvb Vx,Hx,Wx,Lx (66),(v1) +-50: vrangeps/d Vx,Hx,Wx,Ib (66),(ev) +-51: vrangess/d Vx,Hx,Wx,Ib (66),(ev) +-54: vfixupimmps/d Vx,Hx,Wx,Ib (66),(ev) +-55: vfixupimmss/d Vx,Hx,Wx,Ib (66),(ev) +-56: vreduceps/d Vx,Wx,Ib (66),(ev) +-57: vreducess/d Vx,Hx,Wx,Ib (66),(ev) +-60: vpcmpestrm Vdq,Wdq,Ib (66),(v1) +-61: vpcmpestri Vdq,Wdq,Ib (66),(v1) +-62: vpcmpistrm Vdq,Wdq,Ib (66),(v1) +-63: vpcmpistri Vdq,Wdq,Ib (66),(v1) +-66: vfpclassps/d Vk,Wx,Ib (66),(ev) +-67: vfpclassss/d Vk,Wx,Ib (66),(ev) +-cc: sha1rnds4 Vdq,Wdq,Ib +-df: VAESKEYGEN Vdq,Wdq,Ib (66),(v1) +-f0: RORX Gy,Ey,Ib (F2),(v) +-EndTable +- +-GrpTable: Grp1 +-0: ADD +-1: OR +-2: ADC +-3: SBB +-4: AND +-5: SUB +-6: XOR +-7: CMP +-EndTable +- +-GrpTable: Grp1A +-0: POP +-EndTable +- +-GrpTable: Grp2 +-0: ROL +-1: ROR +-2: RCL +-3: RCR +-4: SHL/SAL +-5: SHR +-6: +-7: SAR +-EndTable +- +-GrpTable: Grp3_1 +-0: TEST Eb,Ib +-1: +-2: NOT Eb +-3: NEG Eb +-4: MUL AL,Eb +-5: IMUL AL,Eb +-6: DIV AL,Eb +-7: IDIV AL,Eb +-EndTable +- +-GrpTable: Grp3_2 +-0: TEST Ev,Iz +-1: +-2: NOT Ev +-3: NEG Ev +-4: MUL rAX,Ev +-5: IMUL rAX,Ev +-6: DIV rAX,Ev +-7: IDIV rAX,Ev +-EndTable +- +-GrpTable: Grp4 +-0: INC Eb +-1: DEC Eb +-EndTable +- +-GrpTable: Grp5 +-0: INC Ev +-1: DEC Ev +-# Note: "forced64" is Intel CPU behavior (see comment about CALL insn). +-2: CALLN Ev (f64) +-3: CALLF Ep +-4: JMPN Ev (f64) +-5: JMPF Mp +-6: PUSH Ev (d64) +-7: +-EndTable +- +-GrpTable: Grp6 +-0: SLDT Rv/Mw +-1: STR Rv/Mw +-2: LLDT Ew +-3: LTR Ew +-4: VERR Ew +-5: VERW Ew +-EndTable +- +-GrpTable: Grp7 +-0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) +-1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) +-2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) +-3: LIDT Ms +-4: SMSW Mw/Rv +-5: rdpkru (110),(11B) | wrpkru (111),(11B) +-6: LMSW Ew +-7: INVLPG Mb | SWAPGS (o64),(000),(11B) | RDTSCP (001),(11B) +-EndTable +- +-GrpTable: Grp8 +-4: BT +-5: BTS +-6: BTR +-7: BTC +-EndTable +- +-GrpTable: Grp9 +-1: CMPXCHG8B/16B Mq/Mdq +-3: xrstors +-4: xsavec +-5: xsaves +-6: VMPTRLD Mq | VMCLEAR Mq (66) | VMXON Mq (F3) | RDRAND Rv (11B) +-7: VMPTRST Mq | VMPTRST Mq (F3) | RDSEED Rv (11B) +-EndTable +- +-GrpTable: Grp10 +-EndTable +- +-# Grp11A and Grp11B are expressed as Grp11 in Intel SDM +-GrpTable: Grp11A +-0: MOV Eb,Ib +-7: XABORT Ib (000),(11B) +-EndTable +- +-GrpTable: Grp11B +-0: MOV Eb,Iz +-7: XBEGIN Jz (000),(11B) +-EndTable +- +-GrpTable: Grp12 +-2: psrlw Nq,Ib (11B) | vpsrlw Hx,Ux,Ib (66),(11B),(v1) +-4: psraw Nq,Ib (11B) | vpsraw Hx,Ux,Ib (66),(11B),(v1) +-6: psllw Nq,Ib (11B) | vpsllw Hx,Ux,Ib (66),(11B),(v1) +-EndTable +- +-GrpTable: Grp13 +-0: vprord/q Hx,Wx,Ib (66),(ev) +-1: vprold/q Hx,Wx,Ib (66),(ev) +-2: psrld Nq,Ib (11B) | vpsrld Hx,Ux,Ib (66),(11B),(v1) +-4: psrad Nq,Ib (11B) | vpsrad Hx,Ux,Ib (66),(11B),(v1) | vpsrad/q Hx,Ux,Ib (66),(evo) +-6: pslld Nq,Ib (11B) | vpslld Hx,Ux,Ib (66),(11B),(v1) +-EndTable +- +-GrpTable: Grp14 +-2: psrlq Nq,Ib (11B) | vpsrlq Hx,Ux,Ib (66),(11B),(v1) +-3: vpsrldq Hx,Ux,Ib (66),(11B),(v1) +-6: psllq Nq,Ib (11B) | vpsllq Hx,Ux,Ib (66),(11B),(v1) +-7: vpslldq Hx,Ux,Ib (66),(11B),(v1) +-EndTable +- +-GrpTable: Grp15 +-0: fxsave | RDFSBASE Ry (F3),(11B) +-1: fxstor | RDGSBASE Ry (F3),(11B) +-2: vldmxcsr Md (v1) | WRFSBASE Ry (F3),(11B) +-3: vstmxcsr Md (v1) | WRGSBASE Ry (F3),(11B) +-4: XSAVE | ptwrite Ey (F3),(11B) +-5: XRSTOR | lfence (11B) +-6: XSAVEOPT | clwb (66) | mfence (11B) +-7: clflush | clflushopt (66) | sfence (11B) +-EndTable +- +-GrpTable: Grp16 +-0: prefetch NTA +-1: prefetch T0 +-2: prefetch T1 +-3: prefetch T2 +-EndTable +- +-GrpTable: Grp17 +-1: BLSR By,Ey (v) +-2: BLSMSK By,Ey (v) +-3: BLSI By,Ey (v) +-EndTable +- +-GrpTable: Grp18 +-1: vgatherpf0dps/d Wx (66),(ev) +-2: vgatherpf1dps/d Wx (66),(ev) +-5: vscatterpf0dps/d Wx (66),(ev) +-6: vscatterpf1dps/d Wx (66),(ev) +-EndTable +- +-GrpTable: Grp19 +-1: vgatherpf0qps/d Wx (66),(ev) +-2: vgatherpf1qps/d Wx (66),(ev) +-5: vscatterpf0qps/d Wx (66),(ev) +-6: vscatterpf1qps/d Wx (66),(ev) +-EndTable +- +-# AMD's Prefetch Group +-GrpTable: GrpP +-0: PREFETCH +-1: PREFETCHW +-EndTable +- +-GrpTable: GrpPDLK +-0: MONTMUL +-1: XSHA1 +-2: XSHA2 +-EndTable +- +-GrpTable: GrpRNG +-0: xstore-rng +-1: xcrypt-ecb +-2: xcrypt-cbc +-4: xcrypt-cfb +-5: xcrypt-ofb +-EndTable +diff --git a/tools/objtool/arch/x86/lib/inat.c b/tools/objtool/arch/x86/lib/inat.c +new file mode 100644 +index 000000000000..c1f01a8e9f65 +--- /dev/null ++++ b/tools/objtool/arch/x86/lib/inat.c +@@ -0,0 +1,97 @@ ++/* ++ * x86 instruction attribute tables ++ * ++ * Written by Masami Hiramatsu ++ * ++ * This program is free software; you can redistribute it and/or modify ++ * it under the terms of the GNU General Public License as published by ++ * the Free Software Foundation; either version 2 of the License, or ++ * (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, write to the Free Software ++ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ++ * ++ */ ++#include ++ ++/* Attribute tables are generated from opcode map */ ++#include "inat-tables.c" ++ ++/* Attribute search APIs */ ++insn_attr_t inat_get_opcode_attribute(insn_byte_t opcode) ++{ ++ return inat_primary_table[opcode]; ++} ++ ++int inat_get_last_prefix_id(insn_byte_t last_pfx) ++{ ++ insn_attr_t lpfx_attr; ++ ++ lpfx_attr = inat_get_opcode_attribute(last_pfx); ++ return inat_last_prefix_id(lpfx_attr); ++} ++ ++insn_attr_t inat_get_escape_attribute(insn_byte_t opcode, int lpfx_id, ++ insn_attr_t esc_attr) ++{ ++ const insn_attr_t *table; ++ int n; ++ ++ n = inat_escape_id(esc_attr); ++ ++ table = inat_escape_tables[n][0]; ++ if (!table) ++ return 0; ++ if (inat_has_variant(table[opcode]) && lpfx_id) { ++ table = inat_escape_tables[n][lpfx_id]; ++ if (!table) ++ return 0; ++ } ++ return table[opcode]; ++} ++ ++insn_attr_t inat_get_group_attribute(insn_byte_t modrm, int lpfx_id, ++ insn_attr_t grp_attr) ++{ ++ const insn_attr_t *table; ++ int n; ++ ++ n = inat_group_id(grp_attr); ++ ++ table = inat_group_tables[n][0]; ++ if (!table) ++ return inat_group_common_attribute(grp_attr); ++ if (inat_has_variant(table[X86_MODRM_REG(modrm)]) && lpfx_id) { ++ table = inat_group_tables[n][lpfx_id]; ++ if (!table) ++ return inat_group_common_attribute(grp_attr); ++ } ++ return table[X86_MODRM_REG(modrm)] | ++ inat_group_common_attribute(grp_attr); ++} ++ ++insn_attr_t inat_get_avx_attribute(insn_byte_t opcode, insn_byte_t vex_m, ++ insn_byte_t vex_p) ++{ ++ const insn_attr_t *table; ++ if (vex_m > X86_VEX_M_MAX || vex_p > INAT_LSTPFX_MAX) ++ return 0; ++ /* At first, this checks the master table */ ++ table = inat_avx_tables[vex_m][0]; ++ if (!table) ++ return 0; ++ if (!inat_is_group(table[opcode]) && vex_p) { ++ /* If this is not a group, get attribute directly */ ++ table = inat_avx_tables[vex_m][vex_p]; ++ if (!table) ++ return 0; ++ } ++ return table[opcode]; ++} ++ +diff --git a/tools/objtool/arch/x86/lib/insn.c b/tools/objtool/arch/x86/lib/insn.c +new file mode 100644 +index 000000000000..1088eb8f3a5f +--- /dev/null ++++ b/tools/objtool/arch/x86/lib/insn.c +@@ -0,0 +1,606 @@ ++/* ++ * x86 instruction analysis ++ * ++ * This program is free software; you can redistribute it and/or modify ++ * it under the terms of the GNU General Public License as published by ++ * the Free Software Foundation; either version 2 of the License, or ++ * (at your option) any later version. ++ * ++ * This program is distributed in the hope that it will be useful, ++ * but WITHOUT ANY WARRANTY; without even the implied warranty of ++ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ++ * GNU General Public License for more details. ++ * ++ * You should have received a copy of the GNU General Public License ++ * along with this program; if not, write to the Free Software ++ * Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. ++ * ++ * Copyright (C) IBM Corporation, 2002, 2004, 2009 ++ */ ++ ++#ifdef __KERNEL__ ++#include ++#else ++#include ++#endif ++#include ++#include ++ ++/* Verify next sizeof(t) bytes can be on the same instruction */ ++#define validate_next(t, insn, n) \ ++ ((insn)->next_byte + sizeof(t) + n <= (insn)->end_kaddr) ++ ++#define __get_next(t, insn) \ ++ ({ t r = *(t*)insn->next_byte; insn->next_byte += sizeof(t); r; }) ++ ++#define __peek_nbyte_next(t, insn, n) \ ++ ({ t r = *(t*)((insn)->next_byte + n); r; }) ++ ++#define get_next(t, insn) \ ++ ({ if (unlikely(!validate_next(t, insn, 0))) goto err_out; __get_next(t, insn); }) ++ ++#define peek_nbyte_next(t, insn, n) \ ++ ({ if (unlikely(!validate_next(t, insn, n))) goto err_out; __peek_nbyte_next(t, insn, n); }) ++ ++#define peek_next(t, insn) peek_nbyte_next(t, insn, 0) ++ ++/** ++ * insn_init() - initialize struct insn ++ * @insn: &struct insn to be initialized ++ * @kaddr: address (in kernel memory) of instruction (or copy thereof) ++ * @x86_64: !0 for 64-bit kernel or 64-bit app ++ */ ++void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64) ++{ ++ /* ++ * Instructions longer than MAX_INSN_SIZE (15 bytes) are invalid ++ * even if the input buffer is long enough to hold them. ++ */ ++ if (buf_len > MAX_INSN_SIZE) ++ buf_len = MAX_INSN_SIZE; ++ ++ memset(insn, 0, sizeof(*insn)); ++ insn->kaddr = kaddr; ++ insn->end_kaddr = kaddr + buf_len; ++ insn->next_byte = kaddr; ++ insn->x86_64 = x86_64 ? 1 : 0; ++ insn->opnd_bytes = 4; ++ if (x86_64) ++ insn->addr_bytes = 8; ++ else ++ insn->addr_bytes = 4; ++} ++ ++/** ++ * insn_get_prefixes - scan x86 instruction prefix bytes ++ * @insn: &struct insn containing instruction ++ * ++ * Populates the @insn->prefixes bitmap, and updates @insn->next_byte ++ * to point to the (first) opcode. No effect if @insn->prefixes.got ++ * is already set. ++ */ ++void insn_get_prefixes(struct insn *insn) ++{ ++ struct insn_field *prefixes = &insn->prefixes; ++ insn_attr_t attr; ++ insn_byte_t b, lb; ++ int i, nb; ++ ++ if (prefixes->got) ++ return; ++ ++ nb = 0; ++ lb = 0; ++ b = peek_next(insn_byte_t, insn); ++ attr = inat_get_opcode_attribute(b); ++ while (inat_is_legacy_prefix(attr)) { ++ /* Skip if same prefix */ ++ for (i = 0; i < nb; i++) ++ if (prefixes->bytes[i] == b) ++ goto found; ++ if (nb == 4) ++ /* Invalid instruction */ ++ break; ++ prefixes->bytes[nb++] = b; ++ if (inat_is_address_size_prefix(attr)) { ++ /* address size switches 2/4 or 4/8 */ ++ if (insn->x86_64) ++ insn->addr_bytes ^= 12; ++ else ++ insn->addr_bytes ^= 6; ++ } else if (inat_is_operand_size_prefix(attr)) { ++ /* oprand size switches 2/4 */ ++ insn->opnd_bytes ^= 6; ++ } ++found: ++ prefixes->nbytes++; ++ insn->next_byte++; ++ lb = b; ++ b = peek_next(insn_byte_t, insn); ++ attr = inat_get_opcode_attribute(b); ++ } ++ /* Set the last prefix */ ++ if (lb && lb != insn->prefixes.bytes[3]) { ++ if (unlikely(insn->prefixes.bytes[3])) { ++ /* Swap the last prefix */ ++ b = insn->prefixes.bytes[3]; ++ for (i = 0; i < nb; i++) ++ if (prefixes->bytes[i] == lb) ++ prefixes->bytes[i] = b; ++ } ++ insn->prefixes.bytes[3] = lb; ++ } ++ ++ /* Decode REX prefix */ ++ if (insn->x86_64) { ++ b = peek_next(insn_byte_t, insn); ++ attr = inat_get_opcode_attribute(b); ++ if (inat_is_rex_prefix(attr)) { ++ insn->rex_prefix.value = b; ++ insn->rex_prefix.nbytes = 1; ++ insn->next_byte++; ++ if (X86_REX_W(b)) ++ /* REX.W overrides opnd_size */ ++ insn->opnd_bytes = 8; ++ } ++ } ++ insn->rex_prefix.got = 1; ++ ++ /* Decode VEX prefix */ ++ b = peek_next(insn_byte_t, insn); ++ attr = inat_get_opcode_attribute(b); ++ if (inat_is_vex_prefix(attr)) { ++ insn_byte_t b2 = peek_nbyte_next(insn_byte_t, insn, 1); ++ if (!insn->x86_64) { ++ /* ++ * In 32-bits mode, if the [7:6] bits (mod bits of ++ * ModRM) on the second byte are not 11b, it is ++ * LDS or LES or BOUND. ++ */ ++ if (X86_MODRM_MOD(b2) != 3) ++ goto vex_end; ++ } ++ insn->vex_prefix.bytes[0] = b; ++ insn->vex_prefix.bytes[1] = b2; ++ if (inat_is_evex_prefix(attr)) { ++ b2 = peek_nbyte_next(insn_byte_t, insn, 2); ++ insn->vex_prefix.bytes[2] = b2; ++ b2 = peek_nbyte_next(insn_byte_t, insn, 3); ++ insn->vex_prefix.bytes[3] = b2; ++ insn->vex_prefix.nbytes = 4; ++ insn->next_byte += 4; ++ if (insn->x86_64 && X86_VEX_W(b2)) ++ /* VEX.W overrides opnd_size */ ++ insn->opnd_bytes = 8; ++ } else if (inat_is_vex3_prefix(attr)) { ++ b2 = peek_nbyte_next(insn_byte_t, insn, 2); ++ insn->vex_prefix.bytes[2] = b2; ++ insn->vex_prefix.nbytes = 3; ++ insn->next_byte += 3; ++ if (insn->x86_64 && X86_VEX_W(b2)) ++ /* VEX.W overrides opnd_size */ ++ insn->opnd_bytes = 8; ++ } else { ++ /* ++ * For VEX2, fake VEX3-like byte#2. ++ * Makes it easier to decode vex.W, vex.vvvv, ++ * vex.L and vex.pp. Masking with 0x7f sets vex.W == 0. ++ */ ++ insn->vex_prefix.bytes[2] = b2 & 0x7f; ++ insn->vex_prefix.nbytes = 2; ++ insn->next_byte += 2; ++ } ++ } ++vex_end: ++ insn->vex_prefix.got = 1; ++ ++ prefixes->got = 1; ++ ++err_out: ++ return; ++} ++ ++/** ++ * insn_get_opcode - collect opcode(s) ++ * @insn: &struct insn containing instruction ++ * ++ * Populates @insn->opcode, updates @insn->next_byte to point past the ++ * opcode byte(s), and set @insn->attr (except for groups). ++ * If necessary, first collects any preceding (prefix) bytes. ++ * Sets @insn->opcode.value = opcode1. No effect if @insn->opcode.got ++ * is already 1. ++ */ ++void insn_get_opcode(struct insn *insn) ++{ ++ struct insn_field *opcode = &insn->opcode; ++ insn_byte_t op; ++ int pfx_id; ++ if (opcode->got) ++ return; ++ if (!insn->prefixes.got) ++ insn_get_prefixes(insn); ++ ++ /* Get first opcode */ ++ op = get_next(insn_byte_t, insn); ++ opcode->bytes[0] = op; ++ opcode->nbytes = 1; ++ ++ /* Check if there is VEX prefix or not */ ++ if (insn_is_avx(insn)) { ++ insn_byte_t m, p; ++ m = insn_vex_m_bits(insn); ++ p = insn_vex_p_bits(insn); ++ insn->attr = inat_get_avx_attribute(op, m, p); ++ if ((inat_must_evex(insn->attr) && !insn_is_evex(insn)) || ++ (!inat_accept_vex(insn->attr) && ++ !inat_is_group(insn->attr))) ++ insn->attr = 0; /* This instruction is bad */ ++ goto end; /* VEX has only 1 byte for opcode */ ++ } ++ ++ insn->attr = inat_get_opcode_attribute(op); ++ while (inat_is_escape(insn->attr)) { ++ /* Get escaped opcode */ ++ op = get_next(insn_byte_t, insn); ++ opcode->bytes[opcode->nbytes++] = op; ++ pfx_id = insn_last_prefix_id(insn); ++ insn->attr = inat_get_escape_attribute(op, pfx_id, insn->attr); ++ } ++ if (inat_must_vex(insn->attr)) ++ insn->attr = 0; /* This instruction is bad */ ++end: ++ opcode->got = 1; ++ ++err_out: ++ return; ++} ++ ++/** ++ * insn_get_modrm - collect ModRM byte, if any ++ * @insn: &struct insn containing instruction ++ * ++ * Populates @insn->modrm and updates @insn->next_byte to point past the ++ * ModRM byte, if any. If necessary, first collects the preceding bytes ++ * (prefixes and opcode(s)). No effect if @insn->modrm.got is already 1. ++ */ ++void insn_get_modrm(struct insn *insn) ++{ ++ struct insn_field *modrm = &insn->modrm; ++ insn_byte_t pfx_id, mod; ++ if (modrm->got) ++ return; ++ if (!insn->opcode.got) ++ insn_get_opcode(insn); ++ ++ if (inat_has_modrm(insn->attr)) { ++ mod = get_next(insn_byte_t, insn); ++ modrm->value = mod; ++ modrm->nbytes = 1; ++ if (inat_is_group(insn->attr)) { ++ pfx_id = insn_last_prefix_id(insn); ++ insn->attr = inat_get_group_attribute(mod, pfx_id, ++ insn->attr); ++ if (insn_is_avx(insn) && !inat_accept_vex(insn->attr)) ++ insn->attr = 0; /* This is bad */ ++ } ++ } ++ ++ if (insn->x86_64 && inat_is_force64(insn->attr)) ++ insn->opnd_bytes = 8; ++ modrm->got = 1; ++ ++err_out: ++ return; ++} ++ ++ ++/** ++ * insn_rip_relative() - Does instruction use RIP-relative addressing mode? ++ * @insn: &struct insn containing instruction ++ * ++ * If necessary, first collects the instruction up to and including the ++ * ModRM byte. No effect if @insn->x86_64 is 0. ++ */ ++int insn_rip_relative(struct insn *insn) ++{ ++ struct insn_field *modrm = &insn->modrm; ++ ++ if (!insn->x86_64) ++ return 0; ++ if (!modrm->got) ++ insn_get_modrm(insn); ++ /* ++ * For rip-relative instructions, the mod field (top 2 bits) ++ * is zero and the r/m field (bottom 3 bits) is 0x5. ++ */ ++ return (modrm->nbytes && (modrm->value & 0xc7) == 0x5); ++} ++ ++/** ++ * insn_get_sib() - Get the SIB byte of instruction ++ * @insn: &struct insn containing instruction ++ * ++ * If necessary, first collects the instruction up to and including the ++ * ModRM byte. ++ */ ++void insn_get_sib(struct insn *insn) ++{ ++ insn_byte_t modrm; ++ ++ if (insn->sib.got) ++ return; ++ if (!insn->modrm.got) ++ insn_get_modrm(insn); ++ if (insn->modrm.nbytes) { ++ modrm = (insn_byte_t)insn->modrm.value; ++ if (insn->addr_bytes != 2 && ++ X86_MODRM_MOD(modrm) != 3 && X86_MODRM_RM(modrm) == 4) { ++ insn->sib.value = get_next(insn_byte_t, insn); ++ insn->sib.nbytes = 1; ++ } ++ } ++ insn->sib.got = 1; ++ ++err_out: ++ return; ++} ++ ++ ++/** ++ * insn_get_displacement() - Get the displacement of instruction ++ * @insn: &struct insn containing instruction ++ * ++ * If necessary, first collects the instruction up to and including the ++ * SIB byte. ++ * Displacement value is sign-expanded. ++ */ ++void insn_get_displacement(struct insn *insn) ++{ ++ insn_byte_t mod, rm, base; ++ ++ if (insn->displacement.got) ++ return; ++ if (!insn->sib.got) ++ insn_get_sib(insn); ++ if (insn->modrm.nbytes) { ++ /* ++ * Interpreting the modrm byte: ++ * mod = 00 - no displacement fields (exceptions below) ++ * mod = 01 - 1-byte displacement field ++ * mod = 10 - displacement field is 4 bytes, or 2 bytes if ++ * address size = 2 (0x67 prefix in 32-bit mode) ++ * mod = 11 - no memory operand ++ * ++ * If address size = 2... ++ * mod = 00, r/m = 110 - displacement field is 2 bytes ++ * ++ * If address size != 2... ++ * mod != 11, r/m = 100 - SIB byte exists ++ * mod = 00, SIB base = 101 - displacement field is 4 bytes ++ * mod = 00, r/m = 101 - rip-relative addressing, displacement ++ * field is 4 bytes ++ */ ++ mod = X86_MODRM_MOD(insn->modrm.value); ++ rm = X86_MODRM_RM(insn->modrm.value); ++ base = X86_SIB_BASE(insn->sib.value); ++ if (mod == 3) ++ goto out; ++ if (mod == 1) { ++ insn->displacement.value = get_next(signed char, insn); ++ insn->displacement.nbytes = 1; ++ } else if (insn->addr_bytes == 2) { ++ if ((mod == 0 && rm == 6) || mod == 2) { ++ insn->displacement.value = ++ get_next(short, insn); ++ insn->displacement.nbytes = 2; ++ } ++ } else { ++ if ((mod == 0 && rm == 5) || mod == 2 || ++ (mod == 0 && base == 5)) { ++ insn->displacement.value = get_next(int, insn); ++ insn->displacement.nbytes = 4; ++ } ++ } ++ } ++out: ++ insn->displacement.got = 1; ++ ++err_out: ++ return; ++} ++ ++/* Decode moffset16/32/64. Return 0 if failed */ ++static int __get_moffset(struct insn *insn) ++{ ++ switch (insn->addr_bytes) { ++ case 2: ++ insn->moffset1.value = get_next(short, insn); ++ insn->moffset1.nbytes = 2; ++ break; ++ case 4: ++ insn->moffset1.value = get_next(int, insn); ++ insn->moffset1.nbytes = 4; ++ break; ++ case 8: ++ insn->moffset1.value = get_next(int, insn); ++ insn->moffset1.nbytes = 4; ++ insn->moffset2.value = get_next(int, insn); ++ insn->moffset2.nbytes = 4; ++ break; ++ default: /* opnd_bytes must be modified manually */ ++ goto err_out; ++ } ++ insn->moffset1.got = insn->moffset2.got = 1; ++ ++ return 1; ++ ++err_out: ++ return 0; ++} ++ ++/* Decode imm v32(Iz). Return 0 if failed */ ++static int __get_immv32(struct insn *insn) ++{ ++ switch (insn->opnd_bytes) { ++ case 2: ++ insn->immediate.value = get_next(short, insn); ++ insn->immediate.nbytes = 2; ++ break; ++ case 4: ++ case 8: ++ insn->immediate.value = get_next(int, insn); ++ insn->immediate.nbytes = 4; ++ break; ++ default: /* opnd_bytes must be modified manually */ ++ goto err_out; ++ } ++ ++ return 1; ++ ++err_out: ++ return 0; ++} ++ ++/* Decode imm v64(Iv/Ov), Return 0 if failed */ ++static int __get_immv(struct insn *insn) ++{ ++ switch (insn->opnd_bytes) { ++ case 2: ++ insn->immediate1.value = get_next(short, insn); ++ insn->immediate1.nbytes = 2; ++ break; ++ case 4: ++ insn->immediate1.value = get_next(int, insn); ++ insn->immediate1.nbytes = 4; ++ break; ++ case 8: ++ insn->immediate1.value = get_next(int, insn); ++ insn->immediate1.nbytes = 4; ++ insn->immediate2.value = get_next(int, insn); ++ insn->immediate2.nbytes = 4; ++ break; ++ default: /* opnd_bytes must be modified manually */ ++ goto err_out; ++ } ++ insn->immediate1.got = insn->immediate2.got = 1; ++ ++ return 1; ++err_out: ++ return 0; ++} ++ ++/* Decode ptr16:16/32(Ap) */ ++static int __get_immptr(struct insn *insn) ++{ ++ switch (insn->opnd_bytes) { ++ case 2: ++ insn->immediate1.value = get_next(short, insn); ++ insn->immediate1.nbytes = 2; ++ break; ++ case 4: ++ insn->immediate1.value = get_next(int, insn); ++ insn->immediate1.nbytes = 4; ++ break; ++ case 8: ++ /* ptr16:64 is not exist (no segment) */ ++ return 0; ++ default: /* opnd_bytes must be modified manually */ ++ goto err_out; ++ } ++ insn->immediate2.value = get_next(unsigned short, insn); ++ insn->immediate2.nbytes = 2; ++ insn->immediate1.got = insn->immediate2.got = 1; ++ ++ return 1; ++err_out: ++ return 0; ++} ++ ++/** ++ * insn_get_immediate() - Get the immediates of instruction ++ * @insn: &struct insn containing instruction ++ * ++ * If necessary, first collects the instruction up to and including the ++ * displacement bytes. ++ * Basically, most of immediates are sign-expanded. Unsigned-value can be ++ * get by bit masking with ((1 << (nbytes * 8)) - 1) ++ */ ++void insn_get_immediate(struct insn *insn) ++{ ++ if (insn->immediate.got) ++ return; ++ if (!insn->displacement.got) ++ insn_get_displacement(insn); ++ ++ if (inat_has_moffset(insn->attr)) { ++ if (!__get_moffset(insn)) ++ goto err_out; ++ goto done; ++ } ++ ++ if (!inat_has_immediate(insn->attr)) ++ /* no immediates */ ++ goto done; ++ ++ switch (inat_immediate_size(insn->attr)) { ++ case INAT_IMM_BYTE: ++ insn->immediate.value = get_next(signed char, insn); ++ insn->immediate.nbytes = 1; ++ break; ++ case INAT_IMM_WORD: ++ insn->immediate.value = get_next(short, insn); ++ insn->immediate.nbytes = 2; ++ break; ++ case INAT_IMM_DWORD: ++ insn->immediate.value = get_next(int, insn); ++ insn->immediate.nbytes = 4; ++ break; ++ case INAT_IMM_QWORD: ++ insn->immediate1.value = get_next(int, insn); ++ insn->immediate1.nbytes = 4; ++ insn->immediate2.value = get_next(int, insn); ++ insn->immediate2.nbytes = 4; ++ break; ++ case INAT_IMM_PTR: ++ if (!__get_immptr(insn)) ++ goto err_out; ++ break; ++ case INAT_IMM_VWORD32: ++ if (!__get_immv32(insn)) ++ goto err_out; ++ break; ++ case INAT_IMM_VWORD: ++ if (!__get_immv(insn)) ++ goto err_out; ++ break; ++ default: ++ /* Here, insn must have an immediate, but failed */ ++ goto err_out; ++ } ++ if (inat_has_second_immediate(insn->attr)) { ++ insn->immediate2.value = get_next(signed char, insn); ++ insn->immediate2.nbytes = 1; ++ } ++done: ++ insn->immediate.got = 1; ++ ++err_out: ++ return; ++} ++ ++/** ++ * insn_get_length() - Get the length of instruction ++ * @insn: &struct insn containing instruction ++ * ++ * If necessary, first collects the instruction up to and including the ++ * immediates bytes. ++ */ ++void insn_get_length(struct insn *insn) ++{ ++ if (insn->length) ++ return; ++ if (!insn->immediate.got) ++ insn_get_immediate(insn); ++ insn->length = (unsigned char)((unsigned long)insn->next_byte ++ - (unsigned long)insn->kaddr); ++} +diff --git a/tools/objtool/arch/x86/lib/x86-opcode-map.txt b/tools/objtool/arch/x86/lib/x86-opcode-map.txt +new file mode 100644 +index 000000000000..e0b85930dd77 +--- /dev/null ++++ b/tools/objtool/arch/x86/lib/x86-opcode-map.txt +@@ -0,0 +1,1072 @@ ++# x86 Opcode Maps ++# ++# This is (mostly) based on following documentations. ++# - Intel(R) 64 and IA-32 Architectures Software Developer's Manual Vol.2C ++# (#326018-047US, June 2013) ++# ++# ++# Table: table-name ++# Referrer: escaped-name ++# AVXcode: avx-code ++# opcode: mnemonic|GrpXXX [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...] ++# (or) ++# opcode: escape # escaped-name ++# EndTable ++# ++# mnemonics that begin with lowercase 'v' accept a VEX or EVEX prefix ++# mnemonics that begin with lowercase 'k' accept a VEX prefix ++# ++# ++# GrpTable: GrpXXX ++# reg: mnemonic [operand1[,operand2...]] [(extra1)[,(extra2)...] [| 2nd-mnemonic ...] ++# EndTable ++# ++# AVX Superscripts ++# (ev): this opcode requires EVEX prefix. ++# (evo): this opcode is changed by EVEX prefix (EVEX opcode) ++# (v): this opcode requires VEX prefix. ++# (v1): this opcode only supports 128bit VEX. ++# ++# Last Prefix Superscripts ++# - (66): the last prefix is 0x66 ++# - (F3): the last prefix is 0xF3 ++# - (F2): the last prefix is 0xF2 ++# - (!F3) : the last prefix is not 0xF3 (including non-last prefix case) ++# - (66&F2): Both 0x66 and 0xF2 prefixes are specified. ++ ++Table: one byte opcode ++Referrer: ++AVXcode: ++# 0x00 - 0x0f ++00: ADD Eb,Gb ++01: ADD Ev,Gv ++02: ADD Gb,Eb ++03: ADD Gv,Ev ++04: ADD AL,Ib ++05: ADD rAX,Iz ++06: PUSH ES (i64) ++07: POP ES (i64) ++08: OR Eb,Gb ++09: OR Ev,Gv ++0a: OR Gb,Eb ++0b: OR Gv,Ev ++0c: OR AL,Ib ++0d: OR rAX,Iz ++0e: PUSH CS (i64) ++0f: escape # 2-byte escape ++# 0x10 - 0x1f ++10: ADC Eb,Gb ++11: ADC Ev,Gv ++12: ADC Gb,Eb ++13: ADC Gv,Ev ++14: ADC AL,Ib ++15: ADC rAX,Iz ++16: PUSH SS (i64) ++17: POP SS (i64) ++18: SBB Eb,Gb ++19: SBB Ev,Gv ++1a: SBB Gb,Eb ++1b: SBB Gv,Ev ++1c: SBB AL,Ib ++1d: SBB rAX,Iz ++1e: PUSH DS (i64) ++1f: POP DS (i64) ++# 0x20 - 0x2f ++20: AND Eb,Gb ++21: AND Ev,Gv ++22: AND Gb,Eb ++23: AND Gv,Ev ++24: AND AL,Ib ++25: AND rAx,Iz ++26: SEG=ES (Prefix) ++27: DAA (i64) ++28: SUB Eb,Gb ++29: SUB Ev,Gv ++2a: SUB Gb,Eb ++2b: SUB Gv,Ev ++2c: SUB AL,Ib ++2d: SUB rAX,Iz ++2e: SEG=CS (Prefix) ++2f: DAS (i64) ++# 0x30 - 0x3f ++30: XOR Eb,Gb ++31: XOR Ev,Gv ++32: XOR Gb,Eb ++33: XOR Gv,Ev ++34: XOR AL,Ib ++35: XOR rAX,Iz ++36: SEG=SS (Prefix) ++37: AAA (i64) ++38: CMP Eb,Gb ++39: CMP Ev,Gv ++3a: CMP Gb,Eb ++3b: CMP Gv,Ev ++3c: CMP AL,Ib ++3d: CMP rAX,Iz ++3e: SEG=DS (Prefix) ++3f: AAS (i64) ++# 0x40 - 0x4f ++40: INC eAX (i64) | REX (o64) ++41: INC eCX (i64) | REX.B (o64) ++42: INC eDX (i64) | REX.X (o64) ++43: INC eBX (i64) | REX.XB (o64) ++44: INC eSP (i64) | REX.R (o64) ++45: INC eBP (i64) | REX.RB (o64) ++46: INC eSI (i64) | REX.RX (o64) ++47: INC eDI (i64) | REX.RXB (o64) ++48: DEC eAX (i64) | REX.W (o64) ++49: DEC eCX (i64) | REX.WB (o64) ++4a: DEC eDX (i64) | REX.WX (o64) ++4b: DEC eBX (i64) | REX.WXB (o64) ++4c: DEC eSP (i64) | REX.WR (o64) ++4d: DEC eBP (i64) | REX.WRB (o64) ++4e: DEC eSI (i64) | REX.WRX (o64) ++4f: DEC eDI (i64) | REX.WRXB (o64) ++# 0x50 - 0x5f ++50: PUSH rAX/r8 (d64) ++51: PUSH rCX/r9 (d64) ++52: PUSH rDX/r10 (d64) ++53: PUSH rBX/r11 (d64) ++54: PUSH rSP/r12 (d64) ++55: PUSH rBP/r13 (d64) ++56: PUSH rSI/r14 (d64) ++57: PUSH rDI/r15 (d64) ++58: POP rAX/r8 (d64) ++59: POP rCX/r9 (d64) ++5a: POP rDX/r10 (d64) ++5b: POP rBX/r11 (d64) ++5c: POP rSP/r12 (d64) ++5d: POP rBP/r13 (d64) ++5e: POP rSI/r14 (d64) ++5f: POP rDI/r15 (d64) ++# 0x60 - 0x6f ++60: PUSHA/PUSHAD (i64) ++61: POPA/POPAD (i64) ++62: BOUND Gv,Ma (i64) | EVEX (Prefix) ++63: ARPL Ew,Gw (i64) | MOVSXD Gv,Ev (o64) ++64: SEG=FS (Prefix) ++65: SEG=GS (Prefix) ++66: Operand-Size (Prefix) ++67: Address-Size (Prefix) ++68: PUSH Iz (d64) ++69: IMUL Gv,Ev,Iz ++6a: PUSH Ib (d64) ++6b: IMUL Gv,Ev,Ib ++6c: INS/INSB Yb,DX ++6d: INS/INSW/INSD Yz,DX ++6e: OUTS/OUTSB DX,Xb ++6f: OUTS/OUTSW/OUTSD DX,Xz ++# 0x70 - 0x7f ++70: JO Jb ++71: JNO Jb ++72: JB/JNAE/JC Jb ++73: JNB/JAE/JNC Jb ++74: JZ/JE Jb ++75: JNZ/JNE Jb ++76: JBE/JNA Jb ++77: JNBE/JA Jb ++78: JS Jb ++79: JNS Jb ++7a: JP/JPE Jb ++7b: JNP/JPO Jb ++7c: JL/JNGE Jb ++7d: JNL/JGE Jb ++7e: JLE/JNG Jb ++7f: JNLE/JG Jb ++# 0x80 - 0x8f ++80: Grp1 Eb,Ib (1A) ++81: Grp1 Ev,Iz (1A) ++82: Grp1 Eb,Ib (1A),(i64) ++83: Grp1 Ev,Ib (1A) ++84: TEST Eb,Gb ++85: TEST Ev,Gv ++86: XCHG Eb,Gb ++87: XCHG Ev,Gv ++88: MOV Eb,Gb ++89: MOV Ev,Gv ++8a: MOV Gb,Eb ++8b: MOV Gv,Ev ++8c: MOV Ev,Sw ++8d: LEA Gv,M ++8e: MOV Sw,Ew ++8f: Grp1A (1A) | POP Ev (d64) ++# 0x90 - 0x9f ++90: NOP | PAUSE (F3) | XCHG r8,rAX ++91: XCHG rCX/r9,rAX ++92: XCHG rDX/r10,rAX ++93: XCHG rBX/r11,rAX ++94: XCHG rSP/r12,rAX ++95: XCHG rBP/r13,rAX ++96: XCHG rSI/r14,rAX ++97: XCHG rDI/r15,rAX ++98: CBW/CWDE/CDQE ++99: CWD/CDQ/CQO ++9a: CALLF Ap (i64) ++9b: FWAIT/WAIT ++9c: PUSHF/D/Q Fv (d64) ++9d: POPF/D/Q Fv (d64) ++9e: SAHF ++9f: LAHF ++# 0xa0 - 0xaf ++a0: MOV AL,Ob ++a1: MOV rAX,Ov ++a2: MOV Ob,AL ++a3: MOV Ov,rAX ++a4: MOVS/B Yb,Xb ++a5: MOVS/W/D/Q Yv,Xv ++a6: CMPS/B Xb,Yb ++a7: CMPS/W/D Xv,Yv ++a8: TEST AL,Ib ++a9: TEST rAX,Iz ++aa: STOS/B Yb,AL ++ab: STOS/W/D/Q Yv,rAX ++ac: LODS/B AL,Xb ++ad: LODS/W/D/Q rAX,Xv ++ae: SCAS/B AL,Yb ++# Note: The May 2011 Intel manual shows Xv for the second parameter of the ++# next instruction but Yv is correct ++af: SCAS/W/D/Q rAX,Yv ++# 0xb0 - 0xbf ++b0: MOV AL/R8L,Ib ++b1: MOV CL/R9L,Ib ++b2: MOV DL/R10L,Ib ++b3: MOV BL/R11L,Ib ++b4: MOV AH/R12L,Ib ++b5: MOV CH/R13L,Ib ++b6: MOV DH/R14L,Ib ++b7: MOV BH/R15L,Ib ++b8: MOV rAX/r8,Iv ++b9: MOV rCX/r9,Iv ++ba: MOV rDX/r10,Iv ++bb: MOV rBX/r11,Iv ++bc: MOV rSP/r12,Iv ++bd: MOV rBP/r13,Iv ++be: MOV rSI/r14,Iv ++bf: MOV rDI/r15,Iv ++# 0xc0 - 0xcf ++c0: Grp2 Eb,Ib (1A) ++c1: Grp2 Ev,Ib (1A) ++c2: RETN Iw (f64) ++c3: RETN ++c4: LES Gz,Mp (i64) | VEX+2byte (Prefix) ++c5: LDS Gz,Mp (i64) | VEX+1byte (Prefix) ++c6: Grp11A Eb,Ib (1A) ++c7: Grp11B Ev,Iz (1A) ++c8: ENTER Iw,Ib ++c9: LEAVE (d64) ++ca: RETF Iw ++cb: RETF ++cc: INT3 ++cd: INT Ib ++ce: INTO (i64) ++cf: IRET/D/Q ++# 0xd0 - 0xdf ++d0: Grp2 Eb,1 (1A) ++d1: Grp2 Ev,1 (1A) ++d2: Grp2 Eb,CL (1A) ++d3: Grp2 Ev,CL (1A) ++d4: AAM Ib (i64) ++d5: AAD Ib (i64) ++d6: ++d7: XLAT/XLATB ++d8: ESC ++d9: ESC ++da: ESC ++db: ESC ++dc: ESC ++dd: ESC ++de: ESC ++df: ESC ++# 0xe0 - 0xef ++# Note: "forced64" is Intel CPU behavior: they ignore 0x66 prefix ++# in 64-bit mode. AMD CPUs accept 0x66 prefix, it causes RIP truncation ++# to 16 bits. In 32-bit mode, 0x66 is accepted by both Intel and AMD. ++e0: LOOPNE/LOOPNZ Jb (f64) ++e1: LOOPE/LOOPZ Jb (f64) ++e2: LOOP Jb (f64) ++e3: JrCXZ Jb (f64) ++e4: IN AL,Ib ++e5: IN eAX,Ib ++e6: OUT Ib,AL ++e7: OUT Ib,eAX ++# With 0x66 prefix in 64-bit mode, for AMD CPUs immediate offset ++# in "near" jumps and calls is 16-bit. For CALL, ++# push of return address is 16-bit wide, RSP is decremented by 2 ++# but is not truncated to 16 bits, unlike RIP. ++e8: CALL Jz (f64) ++e9: JMP-near Jz (f64) ++ea: JMP-far Ap (i64) ++eb: JMP-short Jb (f64) ++ec: IN AL,DX ++ed: IN eAX,DX ++ee: OUT DX,AL ++ef: OUT DX,eAX ++# 0xf0 - 0xff ++f0: LOCK (Prefix) ++f1: ++f2: REPNE (Prefix) | XACQUIRE (Prefix) ++f3: REP/REPE (Prefix) | XRELEASE (Prefix) ++f4: HLT ++f5: CMC ++f6: Grp3_1 Eb (1A) ++f7: Grp3_2 Ev (1A) ++f8: CLC ++f9: STC ++fa: CLI ++fb: STI ++fc: CLD ++fd: STD ++fe: Grp4 (1A) ++ff: Grp5 (1A) ++EndTable ++ ++Table: 2-byte opcode (0x0f) ++Referrer: 2-byte escape ++AVXcode: 1 ++# 0x0f 0x00-0x0f ++00: Grp6 (1A) ++01: Grp7 (1A) ++02: LAR Gv,Ew ++03: LSL Gv,Ew ++04: ++05: SYSCALL (o64) ++06: CLTS ++07: SYSRET (o64) ++08: INVD ++09: WBINVD ++0a: ++0b: UD2 (1B) ++0c: ++# AMD's prefetch group. Intel supports prefetchw(/1) only. ++0d: GrpP ++0e: FEMMS ++# 3DNow! uses the last imm byte as opcode extension. ++0f: 3DNow! Pq,Qq,Ib ++# 0x0f 0x10-0x1f ++# NOTE: According to Intel SDM opcode map, vmovups and vmovupd has no operands ++# but it actually has operands. And also, vmovss and vmovsd only accept 128bit. ++# MOVSS/MOVSD has too many forms(3) on SDM. This map just shows a typical form. ++# Many AVX instructions lack v1 superscript, according to Intel AVX-Prgramming ++# Reference A.1 ++10: vmovups Vps,Wps | vmovupd Vpd,Wpd (66) | vmovss Vx,Hx,Wss (F3),(v1) | vmovsd Vx,Hx,Wsd (F2),(v1) ++11: vmovups Wps,Vps | vmovupd Wpd,Vpd (66) | vmovss Wss,Hx,Vss (F3),(v1) | vmovsd Wsd,Hx,Vsd (F2),(v1) ++12: vmovlps Vq,Hq,Mq (v1) | vmovhlps Vq,Hq,Uq (v1) | vmovlpd Vq,Hq,Mq (66),(v1) | vmovsldup Vx,Wx (F3) | vmovddup Vx,Wx (F2) ++13: vmovlps Mq,Vq (v1) | vmovlpd Mq,Vq (66),(v1) ++14: vunpcklps Vx,Hx,Wx | vunpcklpd Vx,Hx,Wx (66) ++15: vunpckhps Vx,Hx,Wx | vunpckhpd Vx,Hx,Wx (66) ++16: vmovhps Vdq,Hq,Mq (v1) | vmovlhps Vdq,Hq,Uq (v1) | vmovhpd Vdq,Hq,Mq (66),(v1) | vmovshdup Vx,Wx (F3) ++17: vmovhps Mq,Vq (v1) | vmovhpd Mq,Vq (66),(v1) ++18: Grp16 (1A) ++19: ++# Intel SDM opcode map does not list MPX instructions. For now using Gv for ++# bnd registers and Ev for everything else is OK because the instruction ++# decoder does not use the information except as an indication that there is ++# a ModR/M byte. ++1a: BNDCL Gv,Ev (F3) | BNDCU Gv,Ev (F2) | BNDMOV Gv,Ev (66) | BNDLDX Gv,Ev ++1b: BNDCN Gv,Ev (F2) | BNDMOV Ev,Gv (66) | BNDMK Gv,Ev (F3) | BNDSTX Ev,Gv ++1c: ++1d: ++1e: ++1f: NOP Ev ++# 0x0f 0x20-0x2f ++20: MOV Rd,Cd ++21: MOV Rd,Dd ++22: MOV Cd,Rd ++23: MOV Dd,Rd ++24: ++25: ++26: ++27: ++28: vmovaps Vps,Wps | vmovapd Vpd,Wpd (66) ++29: vmovaps Wps,Vps | vmovapd Wpd,Vpd (66) ++2a: cvtpi2ps Vps,Qpi | cvtpi2pd Vpd,Qpi (66) | vcvtsi2ss Vss,Hss,Ey (F3),(v1) | vcvtsi2sd Vsd,Hsd,Ey (F2),(v1) ++2b: vmovntps Mps,Vps | vmovntpd Mpd,Vpd (66) ++2c: cvttps2pi Ppi,Wps | cvttpd2pi Ppi,Wpd (66) | vcvttss2si Gy,Wss (F3),(v1) | vcvttsd2si Gy,Wsd (F2),(v1) ++2d: cvtps2pi Ppi,Wps | cvtpd2pi Qpi,Wpd (66) | vcvtss2si Gy,Wss (F3),(v1) | vcvtsd2si Gy,Wsd (F2),(v1) ++2e: vucomiss Vss,Wss (v1) | vucomisd Vsd,Wsd (66),(v1) ++2f: vcomiss Vss,Wss (v1) | vcomisd Vsd,Wsd (66),(v1) ++# 0x0f 0x30-0x3f ++30: WRMSR ++31: RDTSC ++32: RDMSR ++33: RDPMC ++34: SYSENTER ++35: SYSEXIT ++36: ++37: GETSEC ++38: escape # 3-byte escape 1 ++39: ++3a: escape # 3-byte escape 2 ++3b: ++3c: ++3d: ++3e: ++3f: ++# 0x0f 0x40-0x4f ++40: CMOVO Gv,Ev ++41: CMOVNO Gv,Ev | kandw/q Vk,Hk,Uk | kandb/d Vk,Hk,Uk (66) ++42: CMOVB/C/NAE Gv,Ev | kandnw/q Vk,Hk,Uk | kandnb/d Vk,Hk,Uk (66) ++43: CMOVAE/NB/NC Gv,Ev ++44: CMOVE/Z Gv,Ev | knotw/q Vk,Uk | knotb/d Vk,Uk (66) ++45: CMOVNE/NZ Gv,Ev | korw/q Vk,Hk,Uk | korb/d Vk,Hk,Uk (66) ++46: CMOVBE/NA Gv,Ev | kxnorw/q Vk,Hk,Uk | kxnorb/d Vk,Hk,Uk (66) ++47: CMOVA/NBE Gv,Ev | kxorw/q Vk,Hk,Uk | kxorb/d Vk,Hk,Uk (66) ++48: CMOVS Gv,Ev ++49: CMOVNS Gv,Ev ++4a: CMOVP/PE Gv,Ev | kaddw/q Vk,Hk,Uk | kaddb/d Vk,Hk,Uk (66) ++4b: CMOVNP/PO Gv,Ev | kunpckbw Vk,Hk,Uk (66) | kunpckwd/dq Vk,Hk,Uk ++4c: CMOVL/NGE Gv,Ev ++4d: CMOVNL/GE Gv,Ev ++4e: CMOVLE/NG Gv,Ev ++4f: CMOVNLE/G Gv,Ev ++# 0x0f 0x50-0x5f ++50: vmovmskps Gy,Ups | vmovmskpd Gy,Upd (66) ++51: vsqrtps Vps,Wps | vsqrtpd Vpd,Wpd (66) | vsqrtss Vss,Hss,Wss (F3),(v1) | vsqrtsd Vsd,Hsd,Wsd (F2),(v1) ++52: vrsqrtps Vps,Wps | vrsqrtss Vss,Hss,Wss (F3),(v1) ++53: vrcpps Vps,Wps | vrcpss Vss,Hss,Wss (F3),(v1) ++54: vandps Vps,Hps,Wps | vandpd Vpd,Hpd,Wpd (66) ++55: vandnps Vps,Hps,Wps | vandnpd Vpd,Hpd,Wpd (66) ++56: vorps Vps,Hps,Wps | vorpd Vpd,Hpd,Wpd (66) ++57: vxorps Vps,Hps,Wps | vxorpd Vpd,Hpd,Wpd (66) ++58: vaddps Vps,Hps,Wps | vaddpd Vpd,Hpd,Wpd (66) | vaddss Vss,Hss,Wss (F3),(v1) | vaddsd Vsd,Hsd,Wsd (F2),(v1) ++59: vmulps Vps,Hps,Wps | vmulpd Vpd,Hpd,Wpd (66) | vmulss Vss,Hss,Wss (F3),(v1) | vmulsd Vsd,Hsd,Wsd (F2),(v1) ++5a: vcvtps2pd Vpd,Wps | vcvtpd2ps Vps,Wpd (66) | vcvtss2sd Vsd,Hx,Wss (F3),(v1) | vcvtsd2ss Vss,Hx,Wsd (F2),(v1) ++5b: vcvtdq2ps Vps,Wdq | vcvtqq2ps Vps,Wqq (evo) | vcvtps2dq Vdq,Wps (66) | vcvttps2dq Vdq,Wps (F3) ++5c: vsubps Vps,Hps,Wps | vsubpd Vpd,Hpd,Wpd (66) | vsubss Vss,Hss,Wss (F3),(v1) | vsubsd Vsd,Hsd,Wsd (F2),(v1) ++5d: vminps Vps,Hps,Wps | vminpd Vpd,Hpd,Wpd (66) | vminss Vss,Hss,Wss (F3),(v1) | vminsd Vsd,Hsd,Wsd (F2),(v1) ++5e: vdivps Vps,Hps,Wps | vdivpd Vpd,Hpd,Wpd (66) | vdivss Vss,Hss,Wss (F3),(v1) | vdivsd Vsd,Hsd,Wsd (F2),(v1) ++5f: vmaxps Vps,Hps,Wps | vmaxpd Vpd,Hpd,Wpd (66) | vmaxss Vss,Hss,Wss (F3),(v1) | vmaxsd Vsd,Hsd,Wsd (F2),(v1) ++# 0x0f 0x60-0x6f ++60: punpcklbw Pq,Qd | vpunpcklbw Vx,Hx,Wx (66),(v1) ++61: punpcklwd Pq,Qd | vpunpcklwd Vx,Hx,Wx (66),(v1) ++62: punpckldq Pq,Qd | vpunpckldq Vx,Hx,Wx (66),(v1) ++63: packsswb Pq,Qq | vpacksswb Vx,Hx,Wx (66),(v1) ++64: pcmpgtb Pq,Qq | vpcmpgtb Vx,Hx,Wx (66),(v1) ++65: pcmpgtw Pq,Qq | vpcmpgtw Vx,Hx,Wx (66),(v1) ++66: pcmpgtd Pq,Qq | vpcmpgtd Vx,Hx,Wx (66),(v1) ++67: packuswb Pq,Qq | vpackuswb Vx,Hx,Wx (66),(v1) ++68: punpckhbw Pq,Qd | vpunpckhbw Vx,Hx,Wx (66),(v1) ++69: punpckhwd Pq,Qd | vpunpckhwd Vx,Hx,Wx (66),(v1) ++6a: punpckhdq Pq,Qd | vpunpckhdq Vx,Hx,Wx (66),(v1) ++6b: packssdw Pq,Qd | vpackssdw Vx,Hx,Wx (66),(v1) ++6c: vpunpcklqdq Vx,Hx,Wx (66),(v1) ++6d: vpunpckhqdq Vx,Hx,Wx (66),(v1) ++6e: movd/q Pd,Ey | vmovd/q Vy,Ey (66),(v1) ++6f: movq Pq,Qq | vmovdqa Vx,Wx (66) | vmovdqa32/64 Vx,Wx (66),(evo) | vmovdqu Vx,Wx (F3) | vmovdqu32/64 Vx,Wx (F3),(evo) | vmovdqu8/16 Vx,Wx (F2),(ev) ++# 0x0f 0x70-0x7f ++70: pshufw Pq,Qq,Ib | vpshufd Vx,Wx,Ib (66),(v1) | vpshufhw Vx,Wx,Ib (F3),(v1) | vpshuflw Vx,Wx,Ib (F2),(v1) ++71: Grp12 (1A) ++72: Grp13 (1A) ++73: Grp14 (1A) ++74: pcmpeqb Pq,Qq | vpcmpeqb Vx,Hx,Wx (66),(v1) ++75: pcmpeqw Pq,Qq | vpcmpeqw Vx,Hx,Wx (66),(v1) ++76: pcmpeqd Pq,Qq | vpcmpeqd Vx,Hx,Wx (66),(v1) ++# Note: Remove (v), because vzeroall and vzeroupper becomes emms without VEX. ++77: emms | vzeroupper | vzeroall ++78: VMREAD Ey,Gy | vcvttps2udq/pd2udq Vx,Wpd (evo) | vcvttsd2usi Gv,Wx (F2),(ev) | vcvttss2usi Gv,Wx (F3),(ev) | vcvttps2uqq/pd2uqq Vx,Wx (66),(ev) ++79: VMWRITE Gy,Ey | vcvtps2udq/pd2udq Vx,Wpd (evo) | vcvtsd2usi Gv,Wx (F2),(ev) | vcvtss2usi Gv,Wx (F3),(ev) | vcvtps2uqq/pd2uqq Vx,Wx (66),(ev) ++7a: vcvtudq2pd/uqq2pd Vpd,Wx (F3),(ev) | vcvtudq2ps/uqq2ps Vpd,Wx (F2),(ev) | vcvttps2qq/pd2qq Vx,Wx (66),(ev) ++7b: vcvtusi2sd Vpd,Hpd,Ev (F2),(ev) | vcvtusi2ss Vps,Hps,Ev (F3),(ev) | vcvtps2qq/pd2qq Vx,Wx (66),(ev) ++7c: vhaddpd Vpd,Hpd,Wpd (66) | vhaddps Vps,Hps,Wps (F2) ++7d: vhsubpd Vpd,Hpd,Wpd (66) | vhsubps Vps,Hps,Wps (F2) ++7e: movd/q Ey,Pd | vmovd/q Ey,Vy (66),(v1) | vmovq Vq,Wq (F3),(v1) ++7f: movq Qq,Pq | vmovdqa Wx,Vx (66) | vmovdqa32/64 Wx,Vx (66),(evo) | vmovdqu Wx,Vx (F3) | vmovdqu32/64 Wx,Vx (F3),(evo) | vmovdqu8/16 Wx,Vx (F2),(ev) ++# 0x0f 0x80-0x8f ++# Note: "forced64" is Intel CPU behavior (see comment about CALL insn). ++80: JO Jz (f64) ++81: JNO Jz (f64) ++82: JB/JC/JNAE Jz (f64) ++83: JAE/JNB/JNC Jz (f64) ++84: JE/JZ Jz (f64) ++85: JNE/JNZ Jz (f64) ++86: JBE/JNA Jz (f64) ++87: JA/JNBE Jz (f64) ++88: JS Jz (f64) ++89: JNS Jz (f64) ++8a: JP/JPE Jz (f64) ++8b: JNP/JPO Jz (f64) ++8c: JL/JNGE Jz (f64) ++8d: JNL/JGE Jz (f64) ++8e: JLE/JNG Jz (f64) ++8f: JNLE/JG Jz (f64) ++# 0x0f 0x90-0x9f ++90: SETO Eb | kmovw/q Vk,Wk | kmovb/d Vk,Wk (66) ++91: SETNO Eb | kmovw/q Mv,Vk | kmovb/d Mv,Vk (66) ++92: SETB/C/NAE Eb | kmovw Vk,Rv | kmovb Vk,Rv (66) | kmovq/d Vk,Rv (F2) ++93: SETAE/NB/NC Eb | kmovw Gv,Uk | kmovb Gv,Uk (66) | kmovq/d Gv,Uk (F2) ++94: SETE/Z Eb ++95: SETNE/NZ Eb ++96: SETBE/NA Eb ++97: SETA/NBE Eb ++98: SETS Eb | kortestw/q Vk,Uk | kortestb/d Vk,Uk (66) ++99: SETNS Eb | ktestw/q Vk,Uk | ktestb/d Vk,Uk (66) ++9a: SETP/PE Eb ++9b: SETNP/PO Eb ++9c: SETL/NGE Eb ++9d: SETNL/GE Eb ++9e: SETLE/NG Eb ++9f: SETNLE/G Eb ++# 0x0f 0xa0-0xaf ++a0: PUSH FS (d64) ++a1: POP FS (d64) ++a2: CPUID ++a3: BT Ev,Gv ++a4: SHLD Ev,Gv,Ib ++a5: SHLD Ev,Gv,CL ++a6: GrpPDLK ++a7: GrpRNG ++a8: PUSH GS (d64) ++a9: POP GS (d64) ++aa: RSM ++ab: BTS Ev,Gv ++ac: SHRD Ev,Gv,Ib ++ad: SHRD Ev,Gv,CL ++ae: Grp15 (1A),(1C) ++af: IMUL Gv,Ev ++# 0x0f 0xb0-0xbf ++b0: CMPXCHG Eb,Gb ++b1: CMPXCHG Ev,Gv ++b2: LSS Gv,Mp ++b3: BTR Ev,Gv ++b4: LFS Gv,Mp ++b5: LGS Gv,Mp ++b6: MOVZX Gv,Eb ++b7: MOVZX Gv,Ew ++b8: JMPE (!F3) | POPCNT Gv,Ev (F3) ++b9: Grp10 (1A) ++ba: Grp8 Ev,Ib (1A) ++bb: BTC Ev,Gv ++bc: BSF Gv,Ev (!F3) | TZCNT Gv,Ev (F3) ++bd: BSR Gv,Ev (!F3) | LZCNT Gv,Ev (F3) ++be: MOVSX Gv,Eb ++bf: MOVSX Gv,Ew ++# 0x0f 0xc0-0xcf ++c0: XADD Eb,Gb ++c1: XADD Ev,Gv ++c2: vcmpps Vps,Hps,Wps,Ib | vcmppd Vpd,Hpd,Wpd,Ib (66) | vcmpss Vss,Hss,Wss,Ib (F3),(v1) | vcmpsd Vsd,Hsd,Wsd,Ib (F2),(v1) ++c3: movnti My,Gy ++c4: pinsrw Pq,Ry/Mw,Ib | vpinsrw Vdq,Hdq,Ry/Mw,Ib (66),(v1) ++c5: pextrw Gd,Nq,Ib | vpextrw Gd,Udq,Ib (66),(v1) ++c6: vshufps Vps,Hps,Wps,Ib | vshufpd Vpd,Hpd,Wpd,Ib (66) ++c7: Grp9 (1A) ++c8: BSWAP RAX/EAX/R8/R8D ++c9: BSWAP RCX/ECX/R9/R9D ++ca: BSWAP RDX/EDX/R10/R10D ++cb: BSWAP RBX/EBX/R11/R11D ++cc: BSWAP RSP/ESP/R12/R12D ++cd: BSWAP RBP/EBP/R13/R13D ++ce: BSWAP RSI/ESI/R14/R14D ++cf: BSWAP RDI/EDI/R15/R15D ++# 0x0f 0xd0-0xdf ++d0: vaddsubpd Vpd,Hpd,Wpd (66) | vaddsubps Vps,Hps,Wps (F2) ++d1: psrlw Pq,Qq | vpsrlw Vx,Hx,Wx (66),(v1) ++d2: psrld Pq,Qq | vpsrld Vx,Hx,Wx (66),(v1) ++d3: psrlq Pq,Qq | vpsrlq Vx,Hx,Wx (66),(v1) ++d4: paddq Pq,Qq | vpaddq Vx,Hx,Wx (66),(v1) ++d5: pmullw Pq,Qq | vpmullw Vx,Hx,Wx (66),(v1) ++d6: vmovq Wq,Vq (66),(v1) | movq2dq Vdq,Nq (F3) | movdq2q Pq,Uq (F2) ++d7: pmovmskb Gd,Nq | vpmovmskb Gd,Ux (66),(v1) ++d8: psubusb Pq,Qq | vpsubusb Vx,Hx,Wx (66),(v1) ++d9: psubusw Pq,Qq | vpsubusw Vx,Hx,Wx (66),(v1) ++da: pminub Pq,Qq | vpminub Vx,Hx,Wx (66),(v1) ++db: pand Pq,Qq | vpand Vx,Hx,Wx (66),(v1) | vpandd/q Vx,Hx,Wx (66),(evo) ++dc: paddusb Pq,Qq | vpaddusb Vx,Hx,Wx (66),(v1) ++dd: paddusw Pq,Qq | vpaddusw Vx,Hx,Wx (66),(v1) ++de: pmaxub Pq,Qq | vpmaxub Vx,Hx,Wx (66),(v1) ++df: pandn Pq,Qq | vpandn Vx,Hx,Wx (66),(v1) | vpandnd/q Vx,Hx,Wx (66),(evo) ++# 0x0f 0xe0-0xef ++e0: pavgb Pq,Qq | vpavgb Vx,Hx,Wx (66),(v1) ++e1: psraw Pq,Qq | vpsraw Vx,Hx,Wx (66),(v1) ++e2: psrad Pq,Qq | vpsrad Vx,Hx,Wx (66),(v1) ++e3: pavgw Pq,Qq | vpavgw Vx,Hx,Wx (66),(v1) ++e4: pmulhuw Pq,Qq | vpmulhuw Vx,Hx,Wx (66),(v1) ++e5: pmulhw Pq,Qq | vpmulhw Vx,Hx,Wx (66),(v1) ++e6: vcvttpd2dq Vx,Wpd (66) | vcvtdq2pd Vx,Wdq (F3) | vcvtdq2pd/qq2pd Vx,Wdq (F3),(evo) | vcvtpd2dq Vx,Wpd (F2) ++e7: movntq Mq,Pq | vmovntdq Mx,Vx (66) ++e8: psubsb Pq,Qq | vpsubsb Vx,Hx,Wx (66),(v1) ++e9: psubsw Pq,Qq | vpsubsw Vx,Hx,Wx (66),(v1) ++ea: pminsw Pq,Qq | vpminsw Vx,Hx,Wx (66),(v1) ++eb: por Pq,Qq | vpor Vx,Hx,Wx (66),(v1) | vpord/q Vx,Hx,Wx (66),(evo) ++ec: paddsb Pq,Qq | vpaddsb Vx,Hx,Wx (66),(v1) ++ed: paddsw Pq,Qq | vpaddsw Vx,Hx,Wx (66),(v1) ++ee: pmaxsw Pq,Qq | vpmaxsw Vx,Hx,Wx (66),(v1) ++ef: pxor Pq,Qq | vpxor Vx,Hx,Wx (66),(v1) | vpxord/q Vx,Hx,Wx (66),(evo) ++# 0x0f 0xf0-0xff ++f0: vlddqu Vx,Mx (F2) ++f1: psllw Pq,Qq | vpsllw Vx,Hx,Wx (66),(v1) ++f2: pslld Pq,Qq | vpslld Vx,Hx,Wx (66),(v1) ++f3: psllq Pq,Qq | vpsllq Vx,Hx,Wx (66),(v1) ++f4: pmuludq Pq,Qq | vpmuludq Vx,Hx,Wx (66),(v1) ++f5: pmaddwd Pq,Qq | vpmaddwd Vx,Hx,Wx (66),(v1) ++f6: psadbw Pq,Qq | vpsadbw Vx,Hx,Wx (66),(v1) ++f7: maskmovq Pq,Nq | vmaskmovdqu Vx,Ux (66),(v1) ++f8: psubb Pq,Qq | vpsubb Vx,Hx,Wx (66),(v1) ++f9: psubw Pq,Qq | vpsubw Vx,Hx,Wx (66),(v1) ++fa: psubd Pq,Qq | vpsubd Vx,Hx,Wx (66),(v1) ++fb: psubq Pq,Qq | vpsubq Vx,Hx,Wx (66),(v1) ++fc: paddb Pq,Qq | vpaddb Vx,Hx,Wx (66),(v1) ++fd: paddw Pq,Qq | vpaddw Vx,Hx,Wx (66),(v1) ++fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1) ++ff: UD0 ++EndTable ++ ++Table: 3-byte opcode 1 (0x0f 0x38) ++Referrer: 3-byte escape 1 ++AVXcode: 2 ++# 0x0f 0x38 0x00-0x0f ++00: pshufb Pq,Qq | vpshufb Vx,Hx,Wx (66),(v1) ++01: phaddw Pq,Qq | vphaddw Vx,Hx,Wx (66),(v1) ++02: phaddd Pq,Qq | vphaddd Vx,Hx,Wx (66),(v1) ++03: phaddsw Pq,Qq | vphaddsw Vx,Hx,Wx (66),(v1) ++04: pmaddubsw Pq,Qq | vpmaddubsw Vx,Hx,Wx (66),(v1) ++05: phsubw Pq,Qq | vphsubw Vx,Hx,Wx (66),(v1) ++06: phsubd Pq,Qq | vphsubd Vx,Hx,Wx (66),(v1) ++07: phsubsw Pq,Qq | vphsubsw Vx,Hx,Wx (66),(v1) ++08: psignb Pq,Qq | vpsignb Vx,Hx,Wx (66),(v1) ++09: psignw Pq,Qq | vpsignw Vx,Hx,Wx (66),(v1) ++0a: psignd Pq,Qq | vpsignd Vx,Hx,Wx (66),(v1) ++0b: pmulhrsw Pq,Qq | vpmulhrsw Vx,Hx,Wx (66),(v1) ++0c: vpermilps Vx,Hx,Wx (66),(v) ++0d: vpermilpd Vx,Hx,Wx (66),(v) ++0e: vtestps Vx,Wx (66),(v) ++0f: vtestpd Vx,Wx (66),(v) ++# 0x0f 0x38 0x10-0x1f ++10: pblendvb Vdq,Wdq (66) | vpsrlvw Vx,Hx,Wx (66),(evo) | vpmovuswb Wx,Vx (F3),(ev) ++11: vpmovusdb Wx,Vd (F3),(ev) | vpsravw Vx,Hx,Wx (66),(ev) ++12: vpmovusqb Wx,Vq (F3),(ev) | vpsllvw Vx,Hx,Wx (66),(ev) ++13: vcvtph2ps Vx,Wx (66),(v) | vpmovusdw Wx,Vd (F3),(ev) ++14: blendvps Vdq,Wdq (66) | vpmovusqw Wx,Vq (F3),(ev) | vprorvd/q Vx,Hx,Wx (66),(evo) ++15: blendvpd Vdq,Wdq (66) | vpmovusqd Wx,Vq (F3),(ev) | vprolvd/q Vx,Hx,Wx (66),(evo) ++16: vpermps Vqq,Hqq,Wqq (66),(v) | vpermps/d Vqq,Hqq,Wqq (66),(evo) ++17: vptest Vx,Wx (66) ++18: vbroadcastss Vx,Wd (66),(v) ++19: vbroadcastsd Vqq,Wq (66),(v) | vbroadcastf32x2 Vqq,Wq (66),(evo) ++1a: vbroadcastf128 Vqq,Mdq (66),(v) | vbroadcastf32x4/64x2 Vqq,Wq (66),(evo) ++1b: vbroadcastf32x8/64x4 Vqq,Mdq (66),(ev) ++1c: pabsb Pq,Qq | vpabsb Vx,Wx (66),(v1) ++1d: pabsw Pq,Qq | vpabsw Vx,Wx (66),(v1) ++1e: pabsd Pq,Qq | vpabsd Vx,Wx (66),(v1) ++1f: vpabsq Vx,Wx (66),(ev) ++# 0x0f 0x38 0x20-0x2f ++20: vpmovsxbw Vx,Ux/Mq (66),(v1) | vpmovswb Wx,Vx (F3),(ev) ++21: vpmovsxbd Vx,Ux/Md (66),(v1) | vpmovsdb Wx,Vd (F3),(ev) ++22: vpmovsxbq Vx,Ux/Mw (66),(v1) | vpmovsqb Wx,Vq (F3),(ev) ++23: vpmovsxwd Vx,Ux/Mq (66),(v1) | vpmovsdw Wx,Vd (F3),(ev) ++24: vpmovsxwq Vx,Ux/Md (66),(v1) | vpmovsqw Wx,Vq (F3),(ev) ++25: vpmovsxdq Vx,Ux/Mq (66),(v1) | vpmovsqd Wx,Vq (F3),(ev) ++26: vptestmb/w Vk,Hx,Wx (66),(ev) | vptestnmb/w Vk,Hx,Wx (F3),(ev) ++27: vptestmd/q Vk,Hx,Wx (66),(ev) | vptestnmd/q Vk,Hx,Wx (F3),(ev) ++28: vpmuldq Vx,Hx,Wx (66),(v1) | vpmovm2b/w Vx,Uk (F3),(ev) ++29: vpcmpeqq Vx,Hx,Wx (66),(v1) | vpmovb2m/w2m Vk,Ux (F3),(ev) ++2a: vmovntdqa Vx,Mx (66),(v1) | vpbroadcastmb2q Vx,Uk (F3),(ev) ++2b: vpackusdw Vx,Hx,Wx (66),(v1) ++2c: vmaskmovps Vx,Hx,Mx (66),(v) | vscalefps/d Vx,Hx,Wx (66),(evo) ++2d: vmaskmovpd Vx,Hx,Mx (66),(v) | vscalefss/d Vx,Hx,Wx (66),(evo) ++2e: vmaskmovps Mx,Hx,Vx (66),(v) ++2f: vmaskmovpd Mx,Hx,Vx (66),(v) ++# 0x0f 0x38 0x30-0x3f ++30: vpmovzxbw Vx,Ux/Mq (66),(v1) | vpmovwb Wx,Vx (F3),(ev) ++31: vpmovzxbd Vx,Ux/Md (66),(v1) | vpmovdb Wx,Vd (F3),(ev) ++32: vpmovzxbq Vx,Ux/Mw (66),(v1) | vpmovqb Wx,Vq (F3),(ev) ++33: vpmovzxwd Vx,Ux/Mq (66),(v1) | vpmovdw Wx,Vd (F3),(ev) ++34: vpmovzxwq Vx,Ux/Md (66),(v1) | vpmovqw Wx,Vq (F3),(ev) ++35: vpmovzxdq Vx,Ux/Mq (66),(v1) | vpmovqd Wx,Vq (F3),(ev) ++36: vpermd Vqq,Hqq,Wqq (66),(v) | vpermd/q Vqq,Hqq,Wqq (66),(evo) ++37: vpcmpgtq Vx,Hx,Wx (66),(v1) ++38: vpminsb Vx,Hx,Wx (66),(v1) | vpmovm2d/q Vx,Uk (F3),(ev) ++39: vpminsd Vx,Hx,Wx (66),(v1) | vpminsd/q Vx,Hx,Wx (66),(evo) | vpmovd2m/q2m Vk,Ux (F3),(ev) ++3a: vpminuw Vx,Hx,Wx (66),(v1) | vpbroadcastmw2d Vx,Uk (F3),(ev) ++3b: vpminud Vx,Hx,Wx (66),(v1) | vpminud/q Vx,Hx,Wx (66),(evo) ++3c: vpmaxsb Vx,Hx,Wx (66),(v1) ++3d: vpmaxsd Vx,Hx,Wx (66),(v1) | vpmaxsd/q Vx,Hx,Wx (66),(evo) ++3e: vpmaxuw Vx,Hx,Wx (66),(v1) ++3f: vpmaxud Vx,Hx,Wx (66),(v1) | vpmaxud/q Vx,Hx,Wx (66),(evo) ++# 0x0f 0x38 0x40-0x8f ++40: vpmulld Vx,Hx,Wx (66),(v1) | vpmulld/q Vx,Hx,Wx (66),(evo) ++41: vphminposuw Vdq,Wdq (66),(v1) ++42: vgetexpps/d Vx,Wx (66),(ev) ++43: vgetexpss/d Vx,Hx,Wx (66),(ev) ++44: vplzcntd/q Vx,Wx (66),(ev) ++45: vpsrlvd/q Vx,Hx,Wx (66),(v) ++46: vpsravd Vx,Hx,Wx (66),(v) | vpsravd/q Vx,Hx,Wx (66),(evo) ++47: vpsllvd/q Vx,Hx,Wx (66),(v) ++# Skip 0x48-0x4b ++4c: vrcp14ps/d Vpd,Wpd (66),(ev) ++4d: vrcp14ss/d Vsd,Hpd,Wsd (66),(ev) ++4e: vrsqrt14ps/d Vpd,Wpd (66),(ev) ++4f: vrsqrt14ss/d Vsd,Hsd,Wsd (66),(ev) ++# Skip 0x50-0x57 ++58: vpbroadcastd Vx,Wx (66),(v) ++59: vpbroadcastq Vx,Wx (66),(v) | vbroadcasti32x2 Vx,Wx (66),(evo) ++5a: vbroadcasti128 Vqq,Mdq (66),(v) | vbroadcasti32x4/64x2 Vx,Wx (66),(evo) ++5b: vbroadcasti32x8/64x4 Vqq,Mdq (66),(ev) ++# Skip 0x5c-0x63 ++64: vpblendmd/q Vx,Hx,Wx (66),(ev) ++65: vblendmps/d Vx,Hx,Wx (66),(ev) ++66: vpblendmb/w Vx,Hx,Wx (66),(ev) ++# Skip 0x67-0x74 ++75: vpermi2b/w Vx,Hx,Wx (66),(ev) ++76: vpermi2d/q Vx,Hx,Wx (66),(ev) ++77: vpermi2ps/d Vx,Hx,Wx (66),(ev) ++78: vpbroadcastb Vx,Wx (66),(v) ++79: vpbroadcastw Vx,Wx (66),(v) ++7a: vpbroadcastb Vx,Rv (66),(ev) ++7b: vpbroadcastw Vx,Rv (66),(ev) ++7c: vpbroadcastd/q Vx,Rv (66),(ev) ++7d: vpermt2b/w Vx,Hx,Wx (66),(ev) ++7e: vpermt2d/q Vx,Hx,Wx (66),(ev) ++7f: vpermt2ps/d Vx,Hx,Wx (66),(ev) ++80: INVEPT Gy,Mdq (66) ++81: INVVPID Gy,Mdq (66) ++82: INVPCID Gy,Mdq (66) ++83: vpmultishiftqb Vx,Hx,Wx (66),(ev) ++88: vexpandps/d Vpd,Wpd (66),(ev) ++89: vpexpandd/q Vx,Wx (66),(ev) ++8a: vcompressps/d Wx,Vx (66),(ev) ++8b: vpcompressd/q Wx,Vx (66),(ev) ++8c: vpmaskmovd/q Vx,Hx,Mx (66),(v) ++8d: vpermb/w Vx,Hx,Wx (66),(ev) ++8e: vpmaskmovd/q Mx,Vx,Hx (66),(v) ++# 0x0f 0x38 0x90-0xbf (FMA) ++90: vgatherdd/q Vx,Hx,Wx (66),(v) | vpgatherdd/q Vx,Wx (66),(evo) ++91: vgatherqd/q Vx,Hx,Wx (66),(v) | vpgatherqd/q Vx,Wx (66),(evo) ++92: vgatherdps/d Vx,Hx,Wx (66),(v) ++93: vgatherqps/d Vx,Hx,Wx (66),(v) ++94: ++95: ++96: vfmaddsub132ps/d Vx,Hx,Wx (66),(v) ++97: vfmsubadd132ps/d Vx,Hx,Wx (66),(v) ++98: vfmadd132ps/d Vx,Hx,Wx (66),(v) ++99: vfmadd132ss/d Vx,Hx,Wx (66),(v),(v1) ++9a: vfmsub132ps/d Vx,Hx,Wx (66),(v) ++9b: vfmsub132ss/d Vx,Hx,Wx (66),(v),(v1) ++9c: vfnmadd132ps/d Vx,Hx,Wx (66),(v) ++9d: vfnmadd132ss/d Vx,Hx,Wx (66),(v),(v1) ++9e: vfnmsub132ps/d Vx,Hx,Wx (66),(v) ++9f: vfnmsub132ss/d Vx,Hx,Wx (66),(v),(v1) ++a0: vpscatterdd/q Wx,Vx (66),(ev) ++a1: vpscatterqd/q Wx,Vx (66),(ev) ++a2: vscatterdps/d Wx,Vx (66),(ev) ++a3: vscatterqps/d Wx,Vx (66),(ev) ++a6: vfmaddsub213ps/d Vx,Hx,Wx (66),(v) ++a7: vfmsubadd213ps/d Vx,Hx,Wx (66),(v) ++a8: vfmadd213ps/d Vx,Hx,Wx (66),(v) ++a9: vfmadd213ss/d Vx,Hx,Wx (66),(v),(v1) ++aa: vfmsub213ps/d Vx,Hx,Wx (66),(v) ++ab: vfmsub213ss/d Vx,Hx,Wx (66),(v),(v1) ++ac: vfnmadd213ps/d Vx,Hx,Wx (66),(v) ++ad: vfnmadd213ss/d Vx,Hx,Wx (66),(v),(v1) ++ae: vfnmsub213ps/d Vx,Hx,Wx (66),(v) ++af: vfnmsub213ss/d Vx,Hx,Wx (66),(v),(v1) ++b4: vpmadd52luq Vx,Hx,Wx (66),(ev) ++b5: vpmadd52huq Vx,Hx,Wx (66),(ev) ++b6: vfmaddsub231ps/d Vx,Hx,Wx (66),(v) ++b7: vfmsubadd231ps/d Vx,Hx,Wx (66),(v) ++b8: vfmadd231ps/d Vx,Hx,Wx (66),(v) ++b9: vfmadd231ss/d Vx,Hx,Wx (66),(v),(v1) ++ba: vfmsub231ps/d Vx,Hx,Wx (66),(v) ++bb: vfmsub231ss/d Vx,Hx,Wx (66),(v),(v1) ++bc: vfnmadd231ps/d Vx,Hx,Wx (66),(v) ++bd: vfnmadd231ss/d Vx,Hx,Wx (66),(v),(v1) ++be: vfnmsub231ps/d Vx,Hx,Wx (66),(v) ++bf: vfnmsub231ss/d Vx,Hx,Wx (66),(v),(v1) ++# 0x0f 0x38 0xc0-0xff ++c4: vpconflictd/q Vx,Wx (66),(ev) ++c6: Grp18 (1A) ++c7: Grp19 (1A) ++c8: sha1nexte Vdq,Wdq | vexp2ps/d Vx,Wx (66),(ev) ++c9: sha1msg1 Vdq,Wdq ++ca: sha1msg2 Vdq,Wdq | vrcp28ps/d Vx,Wx (66),(ev) ++cb: sha256rnds2 Vdq,Wdq | vrcp28ss/d Vx,Hx,Wx (66),(ev) ++cc: sha256msg1 Vdq,Wdq | vrsqrt28ps/d Vx,Wx (66),(ev) ++cd: sha256msg2 Vdq,Wdq | vrsqrt28ss/d Vx,Hx,Wx (66),(ev) ++db: VAESIMC Vdq,Wdq (66),(v1) ++dc: VAESENC Vdq,Hdq,Wdq (66),(v1) ++dd: VAESENCLAST Vdq,Hdq,Wdq (66),(v1) ++de: VAESDEC Vdq,Hdq,Wdq (66),(v1) ++df: VAESDECLAST Vdq,Hdq,Wdq (66),(v1) ++f0: MOVBE Gy,My | MOVBE Gw,Mw (66) | CRC32 Gd,Eb (F2) | CRC32 Gd,Eb (66&F2) ++f1: MOVBE My,Gy | MOVBE Mw,Gw (66) | CRC32 Gd,Ey (F2) | CRC32 Gd,Ew (66&F2) ++f2: ANDN Gy,By,Ey (v) ++f3: Grp17 (1A) ++f5: BZHI Gy,Ey,By (v) | PEXT Gy,By,Ey (F3),(v) | PDEP Gy,By,Ey (F2),(v) ++f6: ADCX Gy,Ey (66) | ADOX Gy,Ey (F3) | MULX By,Gy,rDX,Ey (F2),(v) ++f7: BEXTR Gy,Ey,By (v) | SHLX Gy,Ey,By (66),(v) | SARX Gy,Ey,By (F3),(v) | SHRX Gy,Ey,By (F2),(v) ++EndTable ++ ++Table: 3-byte opcode 2 (0x0f 0x3a) ++Referrer: 3-byte escape 2 ++AVXcode: 3 ++# 0x0f 0x3a 0x00-0xff ++00: vpermq Vqq,Wqq,Ib (66),(v) ++01: vpermpd Vqq,Wqq,Ib (66),(v) ++02: vpblendd Vx,Hx,Wx,Ib (66),(v) ++03: valignd/q Vx,Hx,Wx,Ib (66),(ev) ++04: vpermilps Vx,Wx,Ib (66),(v) ++05: vpermilpd Vx,Wx,Ib (66),(v) ++06: vperm2f128 Vqq,Hqq,Wqq,Ib (66),(v) ++07: ++08: vroundps Vx,Wx,Ib (66) | vrndscaleps Vx,Wx,Ib (66),(evo) ++09: vroundpd Vx,Wx,Ib (66) | vrndscalepd Vx,Wx,Ib (66),(evo) ++0a: vroundss Vss,Wss,Ib (66),(v1) | vrndscaless Vx,Hx,Wx,Ib (66),(evo) ++0b: vroundsd Vsd,Wsd,Ib (66),(v1) | vrndscalesd Vx,Hx,Wx,Ib (66),(evo) ++0c: vblendps Vx,Hx,Wx,Ib (66) ++0d: vblendpd Vx,Hx,Wx,Ib (66) ++0e: vpblendw Vx,Hx,Wx,Ib (66),(v1) ++0f: palignr Pq,Qq,Ib | vpalignr Vx,Hx,Wx,Ib (66),(v1) ++14: vpextrb Rd/Mb,Vdq,Ib (66),(v1) ++15: vpextrw Rd/Mw,Vdq,Ib (66),(v1) ++16: vpextrd/q Ey,Vdq,Ib (66),(v1) ++17: vextractps Ed,Vdq,Ib (66),(v1) ++18: vinsertf128 Vqq,Hqq,Wqq,Ib (66),(v) | vinsertf32x4/64x2 Vqq,Hqq,Wqq,Ib (66),(evo) ++19: vextractf128 Wdq,Vqq,Ib (66),(v) | vextractf32x4/64x2 Wdq,Vqq,Ib (66),(evo) ++1a: vinsertf32x8/64x4 Vqq,Hqq,Wqq,Ib (66),(ev) ++1b: vextractf32x8/64x4 Wdq,Vqq,Ib (66),(ev) ++1d: vcvtps2ph Wx,Vx,Ib (66),(v) ++1e: vpcmpud/q Vk,Hd,Wd,Ib (66),(ev) ++1f: vpcmpd/q Vk,Hd,Wd,Ib (66),(ev) ++20: vpinsrb Vdq,Hdq,Ry/Mb,Ib (66),(v1) ++21: vinsertps Vdq,Hdq,Udq/Md,Ib (66),(v1) ++22: vpinsrd/q Vdq,Hdq,Ey,Ib (66),(v1) ++23: vshuff32x4/64x2 Vx,Hx,Wx,Ib (66),(ev) ++25: vpternlogd/q Vx,Hx,Wx,Ib (66),(ev) ++26: vgetmantps/d Vx,Wx,Ib (66),(ev) ++27: vgetmantss/d Vx,Hx,Wx,Ib (66),(ev) ++30: kshiftrb/w Vk,Uk,Ib (66),(v) ++31: kshiftrd/q Vk,Uk,Ib (66),(v) ++32: kshiftlb/w Vk,Uk,Ib (66),(v) ++33: kshiftld/q Vk,Uk,Ib (66),(v) ++38: vinserti128 Vqq,Hqq,Wqq,Ib (66),(v) | vinserti32x4/64x2 Vqq,Hqq,Wqq,Ib (66),(evo) ++39: vextracti128 Wdq,Vqq,Ib (66),(v) | vextracti32x4/64x2 Wdq,Vqq,Ib (66),(evo) ++3a: vinserti32x8/64x4 Vqq,Hqq,Wqq,Ib (66),(ev) ++3b: vextracti32x8/64x4 Wdq,Vqq,Ib (66),(ev) ++3e: vpcmpub/w Vk,Hk,Wx,Ib (66),(ev) ++3f: vpcmpb/w Vk,Hk,Wx,Ib (66),(ev) ++40: vdpps Vx,Hx,Wx,Ib (66) ++41: vdppd Vdq,Hdq,Wdq,Ib (66),(v1) ++42: vmpsadbw Vx,Hx,Wx,Ib (66),(v1) | vdbpsadbw Vx,Hx,Wx,Ib (66),(evo) ++43: vshufi32x4/64x2 Vx,Hx,Wx,Ib (66),(ev) ++44: vpclmulqdq Vdq,Hdq,Wdq,Ib (66),(v1) ++46: vperm2i128 Vqq,Hqq,Wqq,Ib (66),(v) ++4a: vblendvps Vx,Hx,Wx,Lx (66),(v) ++4b: vblendvpd Vx,Hx,Wx,Lx (66),(v) ++4c: vpblendvb Vx,Hx,Wx,Lx (66),(v1) ++50: vrangeps/d Vx,Hx,Wx,Ib (66),(ev) ++51: vrangess/d Vx,Hx,Wx,Ib (66),(ev) ++54: vfixupimmps/d Vx,Hx,Wx,Ib (66),(ev) ++55: vfixupimmss/d Vx,Hx,Wx,Ib (66),(ev) ++56: vreduceps/d Vx,Wx,Ib (66),(ev) ++57: vreducess/d Vx,Hx,Wx,Ib (66),(ev) ++60: vpcmpestrm Vdq,Wdq,Ib (66),(v1) ++61: vpcmpestri Vdq,Wdq,Ib (66),(v1) ++62: vpcmpistrm Vdq,Wdq,Ib (66),(v1) ++63: vpcmpistri Vdq,Wdq,Ib (66),(v1) ++66: vfpclassps/d Vk,Wx,Ib (66),(ev) ++67: vfpclassss/d Vk,Wx,Ib (66),(ev) ++cc: sha1rnds4 Vdq,Wdq,Ib ++df: VAESKEYGEN Vdq,Wdq,Ib (66),(v1) ++f0: RORX Gy,Ey,Ib (F2),(v) ++EndTable ++ ++GrpTable: Grp1 ++0: ADD ++1: OR ++2: ADC ++3: SBB ++4: AND ++5: SUB ++6: XOR ++7: CMP ++EndTable ++ ++GrpTable: Grp1A ++0: POP ++EndTable ++ ++GrpTable: Grp2 ++0: ROL ++1: ROR ++2: RCL ++3: RCR ++4: SHL/SAL ++5: SHR ++6: ++7: SAR ++EndTable ++ ++GrpTable: Grp3_1 ++0: TEST Eb,Ib ++1: TEST Eb,Ib ++2: NOT Eb ++3: NEG Eb ++4: MUL AL,Eb ++5: IMUL AL,Eb ++6: DIV AL,Eb ++7: IDIV AL,Eb ++EndTable ++ ++GrpTable: Grp3_2 ++0: TEST Ev,Iz ++1: ++2: NOT Ev ++3: NEG Ev ++4: MUL rAX,Ev ++5: IMUL rAX,Ev ++6: DIV rAX,Ev ++7: IDIV rAX,Ev ++EndTable ++ ++GrpTable: Grp4 ++0: INC Eb ++1: DEC Eb ++EndTable ++ ++GrpTable: Grp5 ++0: INC Ev ++1: DEC Ev ++# Note: "forced64" is Intel CPU behavior (see comment about CALL insn). ++2: CALLN Ev (f64) ++3: CALLF Ep ++4: JMPN Ev (f64) ++5: JMPF Mp ++6: PUSH Ev (d64) ++7: ++EndTable ++ ++GrpTable: Grp6 ++0: SLDT Rv/Mw ++1: STR Rv/Mw ++2: LLDT Ew ++3: LTR Ew ++4: VERR Ew ++5: VERW Ew ++EndTable ++ ++GrpTable: Grp7 ++0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) ++1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) ++2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) ++3: LIDT Ms ++4: SMSW Mw/Rv ++5: rdpkru (110),(11B) | wrpkru (111),(11B) ++6: LMSW Ew ++7: INVLPG Mb | SWAPGS (o64),(000),(11B) | RDTSCP (001),(11B) ++EndTable ++ ++GrpTable: Grp8 ++4: BT ++5: BTS ++6: BTR ++7: BTC ++EndTable ++ ++GrpTable: Grp9 ++1: CMPXCHG8B/16B Mq/Mdq ++3: xrstors ++4: xsavec ++5: xsaves ++6: VMPTRLD Mq | VMCLEAR Mq (66) | VMXON Mq (F3) | RDRAND Rv (11B) ++7: VMPTRST Mq | VMPTRST Mq (F3) | RDSEED Rv (11B) ++EndTable ++ ++GrpTable: Grp10 ++# all are UD1 ++0: UD1 ++1: UD1 ++2: UD1 ++3: UD1 ++4: UD1 ++5: UD1 ++6: UD1 ++7: UD1 ++EndTable ++ ++# Grp11A and Grp11B are expressed as Grp11 in Intel SDM ++GrpTable: Grp11A ++0: MOV Eb,Ib ++7: XABORT Ib (000),(11B) ++EndTable ++ ++GrpTable: Grp11B ++0: MOV Eb,Iz ++7: XBEGIN Jz (000),(11B) ++EndTable ++ ++GrpTable: Grp12 ++2: psrlw Nq,Ib (11B) | vpsrlw Hx,Ux,Ib (66),(11B),(v1) ++4: psraw Nq,Ib (11B) | vpsraw Hx,Ux,Ib (66),(11B),(v1) ++6: psllw Nq,Ib (11B) | vpsllw Hx,Ux,Ib (66),(11B),(v1) ++EndTable ++ ++GrpTable: Grp13 ++0: vprord/q Hx,Wx,Ib (66),(ev) ++1: vprold/q Hx,Wx,Ib (66),(ev) ++2: psrld Nq,Ib (11B) | vpsrld Hx,Ux,Ib (66),(11B),(v1) ++4: psrad Nq,Ib (11B) | vpsrad Hx,Ux,Ib (66),(11B),(v1) | vpsrad/q Hx,Ux,Ib (66),(evo) ++6: pslld Nq,Ib (11B) | vpslld Hx,Ux,Ib (66),(11B),(v1) ++EndTable ++ ++GrpTable: Grp14 ++2: psrlq Nq,Ib (11B) | vpsrlq Hx,Ux,Ib (66),(11B),(v1) ++3: vpsrldq Hx,Ux,Ib (66),(11B),(v1) ++6: psllq Nq,Ib (11B) | vpsllq Hx,Ux,Ib (66),(11B),(v1) ++7: vpslldq Hx,Ux,Ib (66),(11B),(v1) ++EndTable ++ ++GrpTable: Grp15 ++0: fxsave | RDFSBASE Ry (F3),(11B) ++1: fxstor | RDGSBASE Ry (F3),(11B) ++2: vldmxcsr Md (v1) | WRFSBASE Ry (F3),(11B) ++3: vstmxcsr Md (v1) | WRGSBASE Ry (F3),(11B) ++4: XSAVE | ptwrite Ey (F3),(11B) ++5: XRSTOR | lfence (11B) ++6: XSAVEOPT | clwb (66) | mfence (11B) ++7: clflush | clflushopt (66) | sfence (11B) ++EndTable ++ ++GrpTable: Grp16 ++0: prefetch NTA ++1: prefetch T0 ++2: prefetch T1 ++3: prefetch T2 ++EndTable ++ ++GrpTable: Grp17 ++1: BLSR By,Ey (v) ++2: BLSMSK By,Ey (v) ++3: BLSI By,Ey (v) ++EndTable ++ ++GrpTable: Grp18 ++1: vgatherpf0dps/d Wx (66),(ev) ++2: vgatherpf1dps/d Wx (66),(ev) ++5: vscatterpf0dps/d Wx (66),(ev) ++6: vscatterpf1dps/d Wx (66),(ev) ++EndTable ++ ++GrpTable: Grp19 ++1: vgatherpf0qps/d Wx (66),(ev) ++2: vgatherpf1qps/d Wx (66),(ev) ++5: vscatterpf0qps/d Wx (66),(ev) ++6: vscatterpf1qps/d Wx (66),(ev) ++EndTable ++ ++# AMD's Prefetch Group ++GrpTable: GrpP ++0: PREFETCH ++1: PREFETCHW ++EndTable ++ ++GrpTable: GrpPDLK ++0: MONTMUL ++1: XSHA1 ++2: XSHA2 ++EndTable ++ ++GrpTable: GrpRNG ++0: xstore-rng ++1: xcrypt-ecb ++2: xcrypt-cbc ++4: xcrypt-cfb ++5: xcrypt-ofb ++EndTable +diff --git a/tools/objtool/arch/x86/tools/gen-insn-attr-x86.awk b/tools/objtool/arch/x86/tools/gen-insn-attr-x86.awk +new file mode 100644 +index 000000000000..b02a36b2c14f +--- /dev/null ++++ b/tools/objtool/arch/x86/tools/gen-insn-attr-x86.awk +@@ -0,0 +1,393 @@ ++#!/bin/awk -f ++# SPDX-License-Identifier: GPL-2.0 ++# gen-insn-attr-x86.awk: Instruction attribute table generator ++# Written by Masami Hiramatsu ++# ++# Usage: awk -f gen-insn-attr-x86.awk x86-opcode-map.txt > inat-tables.c ++ ++# Awk implementation sanity check ++function check_awk_implement() { ++ if (sprintf("%x", 0) != "0") ++ return "Your awk has a printf-format problem." ++ return "" ++} ++ ++# Clear working vars ++function clear_vars() { ++ delete table ++ delete lptable2 ++ delete lptable1 ++ delete lptable3 ++ eid = -1 # escape id ++ gid = -1 # group id ++ aid = -1 # AVX id ++ tname = "" ++} ++ ++BEGIN { ++ # Implementation error checking ++ awkchecked = check_awk_implement() ++ if (awkchecked != "") { ++ print "Error: " awkchecked > "/dev/stderr" ++ print "Please try to use gawk." > "/dev/stderr" ++ exit 1 ++ } ++ ++ # Setup generating tables ++ print "/* x86 opcode map generated from x86-opcode-map.txt */" ++ print "/* Do not change this code. */\n" ++ ggid = 1 ++ geid = 1 ++ gaid = 0 ++ delete etable ++ delete gtable ++ delete atable ++ ++ opnd_expr = "^[A-Za-z/]" ++ ext_expr = "^\\(" ++ sep_expr = "^\\|$" ++ group_expr = "^Grp[0-9A-Za-z]+" ++ ++ imm_expr = "^[IJAOL][a-z]" ++ imm_flag["Ib"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" ++ imm_flag["Jb"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" ++ imm_flag["Iw"] = "INAT_MAKE_IMM(INAT_IMM_WORD)" ++ imm_flag["Id"] = "INAT_MAKE_IMM(INAT_IMM_DWORD)" ++ imm_flag["Iq"] = "INAT_MAKE_IMM(INAT_IMM_QWORD)" ++ imm_flag["Ap"] = "INAT_MAKE_IMM(INAT_IMM_PTR)" ++ imm_flag["Iz"] = "INAT_MAKE_IMM(INAT_IMM_VWORD32)" ++ imm_flag["Jz"] = "INAT_MAKE_IMM(INAT_IMM_VWORD32)" ++ imm_flag["Iv"] = "INAT_MAKE_IMM(INAT_IMM_VWORD)" ++ imm_flag["Ob"] = "INAT_MOFFSET" ++ imm_flag["Ov"] = "INAT_MOFFSET" ++ imm_flag["Lx"] = "INAT_MAKE_IMM(INAT_IMM_BYTE)" ++ ++ modrm_expr = "^([CDEGMNPQRSUVW/][a-z]+|NTA|T[012])" ++ force64_expr = "\\([df]64\\)" ++ rex_expr = "^REX(\\.[XRWB]+)*" ++ fpu_expr = "^ESC" # TODO ++ ++ lprefix1_expr = "\\((66|!F3)\\)" ++ lprefix2_expr = "\\(F3\\)" ++ lprefix3_expr = "\\((F2|!F3|66\\&F2)\\)" ++ lprefix_expr = "\\((66|F2|F3)\\)" ++ max_lprefix = 4 ++ ++ # All opcodes starting with lower-case 'v', 'k' or with (v1) superscript ++ # accepts VEX prefix ++ vexok_opcode_expr = "^[vk].*" ++ vexok_expr = "\\(v1\\)" ++ # All opcodes with (v) superscript supports *only* VEX prefix ++ vexonly_expr = "\\(v\\)" ++ # All opcodes with (ev) superscript supports *only* EVEX prefix ++ evexonly_expr = "\\(ev\\)" ++ ++ prefix_expr = "\\(Prefix\\)" ++ prefix_num["Operand-Size"] = "INAT_PFX_OPNDSZ" ++ prefix_num["REPNE"] = "INAT_PFX_REPNE" ++ prefix_num["REP/REPE"] = "INAT_PFX_REPE" ++ prefix_num["XACQUIRE"] = "INAT_PFX_REPNE" ++ prefix_num["XRELEASE"] = "INAT_PFX_REPE" ++ prefix_num["LOCK"] = "INAT_PFX_LOCK" ++ prefix_num["SEG=CS"] = "INAT_PFX_CS" ++ prefix_num["SEG=DS"] = "INAT_PFX_DS" ++ prefix_num["SEG=ES"] = "INAT_PFX_ES" ++ prefix_num["SEG=FS"] = "INAT_PFX_FS" ++ prefix_num["SEG=GS"] = "INAT_PFX_GS" ++ prefix_num["SEG=SS"] = "INAT_PFX_SS" ++ prefix_num["Address-Size"] = "INAT_PFX_ADDRSZ" ++ prefix_num["VEX+1byte"] = "INAT_PFX_VEX2" ++ prefix_num["VEX+2byte"] = "INAT_PFX_VEX3" ++ prefix_num["EVEX"] = "INAT_PFX_EVEX" ++ ++ clear_vars() ++} ++ ++function semantic_error(msg) { ++ print "Semantic error at " NR ": " msg > "/dev/stderr" ++ exit 1 ++} ++ ++function debug(msg) { ++ print "DEBUG: " msg ++} ++ ++function array_size(arr, i,c) { ++ c = 0 ++ for (i in arr) ++ c++ ++ return c ++} ++ ++/^Table:/ { ++ print "/* " $0 " */" ++ if (tname != "") ++ semantic_error("Hit Table: before EndTable:."); ++} ++ ++/^Referrer:/ { ++ if (NF != 1) { ++ # escape opcode table ++ ref = "" ++ for (i = 2; i <= NF; i++) ++ ref = ref $i ++ eid = escape[ref] ++ tname = sprintf("inat_escape_table_%d", eid) ++ } ++} ++ ++/^AVXcode:/ { ++ if (NF != 1) { ++ # AVX/escape opcode table ++ aid = $2 ++ if (gaid <= aid) ++ gaid = aid + 1 ++ if (tname == "") # AVX only opcode table ++ tname = sprintf("inat_avx_table_%d", $2) ++ } ++ if (aid == -1 && eid == -1) # primary opcode table ++ tname = "inat_primary_table" ++} ++ ++/^GrpTable:/ { ++ print "/* " $0 " */" ++ if (!($2 in group)) ++ semantic_error("No group: " $2 ) ++ gid = group[$2] ++ tname = "inat_group_table_" gid ++} ++ ++function print_table(tbl,name,fmt,n) ++{ ++ print "const insn_attr_t " name " = {" ++ for (i = 0; i < n; i++) { ++ id = sprintf(fmt, i) ++ if (tbl[id]) ++ print " [" id "] = " tbl[id] "," ++ } ++ print "};" ++} ++ ++/^EndTable/ { ++ if (gid != -1) { ++ # print group tables ++ if (array_size(table) != 0) { ++ print_table(table, tname "[INAT_GROUP_TABLE_SIZE]", ++ "0x%x", 8) ++ gtable[gid,0] = tname ++ } ++ if (array_size(lptable1) != 0) { ++ print_table(lptable1, tname "_1[INAT_GROUP_TABLE_SIZE]", ++ "0x%x", 8) ++ gtable[gid,1] = tname "_1" ++ } ++ if (array_size(lptable2) != 0) { ++ print_table(lptable2, tname "_2[INAT_GROUP_TABLE_SIZE]", ++ "0x%x", 8) ++ gtable[gid,2] = tname "_2" ++ } ++ if (array_size(lptable3) != 0) { ++ print_table(lptable3, tname "_3[INAT_GROUP_TABLE_SIZE]", ++ "0x%x", 8) ++ gtable[gid,3] = tname "_3" ++ } ++ } else { ++ # print primary/escaped tables ++ if (array_size(table) != 0) { ++ print_table(table, tname "[INAT_OPCODE_TABLE_SIZE]", ++ "0x%02x", 256) ++ etable[eid,0] = tname ++ if (aid >= 0) ++ atable[aid,0] = tname ++ } ++ if (array_size(lptable1) != 0) { ++ print_table(lptable1,tname "_1[INAT_OPCODE_TABLE_SIZE]", ++ "0x%02x", 256) ++ etable[eid,1] = tname "_1" ++ if (aid >= 0) ++ atable[aid,1] = tname "_1" ++ } ++ if (array_size(lptable2) != 0) { ++ print_table(lptable2,tname "_2[INAT_OPCODE_TABLE_SIZE]", ++ "0x%02x", 256) ++ etable[eid,2] = tname "_2" ++ if (aid >= 0) ++ atable[aid,2] = tname "_2" ++ } ++ if (array_size(lptable3) != 0) { ++ print_table(lptable3,tname "_3[INAT_OPCODE_TABLE_SIZE]", ++ "0x%02x", 256) ++ etable[eid,3] = tname "_3" ++ if (aid >= 0) ++ atable[aid,3] = tname "_3" ++ } ++ } ++ print "" ++ clear_vars() ++} ++ ++function add_flags(old,new) { ++ if (old && new) ++ return old " | " new ++ else if (old) ++ return old ++ else ++ return new ++} ++ ++# convert operands to flags. ++function convert_operands(count,opnd, i,j,imm,mod) ++{ ++ imm = null ++ mod = null ++ for (j = 1; j <= count; j++) { ++ i = opnd[j] ++ if (match(i, imm_expr) == 1) { ++ if (!imm_flag[i]) ++ semantic_error("Unknown imm opnd: " i) ++ if (imm) { ++ if (i != "Ib") ++ semantic_error("Second IMM error") ++ imm = add_flags(imm, "INAT_SCNDIMM") ++ } else ++ imm = imm_flag[i] ++ } else if (match(i, modrm_expr)) ++ mod = "INAT_MODRM" ++ } ++ return add_flags(imm, mod) ++} ++ ++/^[0-9a-f]+\:/ { ++ if (NR == 1) ++ next ++ # get index ++ idx = "0x" substr($1, 1, index($1,":") - 1) ++ if (idx in table) ++ semantic_error("Redefine " idx " in " tname) ++ ++ # check if escaped opcode ++ if ("escape" == $2) { ++ if ($3 != "#") ++ semantic_error("No escaped name") ++ ref = "" ++ for (i = 4; i <= NF; i++) ++ ref = ref $i ++ if (ref in escape) ++ semantic_error("Redefine escape (" ref ")") ++ escape[ref] = geid ++ geid++ ++ table[idx] = "INAT_MAKE_ESCAPE(" escape[ref] ")" ++ next ++ } ++ ++ variant = null ++ # converts ++ i = 2 ++ while (i <= NF) { ++ opcode = $(i++) ++ delete opnds ++ ext = null ++ flags = null ++ opnd = null ++ # parse one opcode ++ if (match($i, opnd_expr)) { ++ opnd = $i ++ count = split($(i++), opnds, ",") ++ flags = convert_operands(count, opnds) ++ } ++ if (match($i, ext_expr)) ++ ext = $(i++) ++ if (match($i, sep_expr)) ++ i++ ++ else if (i < NF) ++ semantic_error($i " is not a separator") ++ ++ # check if group opcode ++ if (match(opcode, group_expr)) { ++ if (!(opcode in group)) { ++ group[opcode] = ggid ++ ggid++ ++ } ++ flags = add_flags(flags, "INAT_MAKE_GROUP(" group[opcode] ")") ++ } ++ # check force(or default) 64bit ++ if (match(ext, force64_expr)) ++ flags = add_flags(flags, "INAT_FORCE64") ++ ++ # check REX prefix ++ if (match(opcode, rex_expr)) ++ flags = add_flags(flags, "INAT_MAKE_PREFIX(INAT_PFX_REX)") ++ ++ # check coprocessor escape : TODO ++ if (match(opcode, fpu_expr)) ++ flags = add_flags(flags, "INAT_MODRM") ++ ++ # check VEX codes ++ if (match(ext, evexonly_expr)) ++ flags = add_flags(flags, "INAT_VEXOK | INAT_EVEXONLY") ++ else if (match(ext, vexonly_expr)) ++ flags = add_flags(flags, "INAT_VEXOK | INAT_VEXONLY") ++ else if (match(ext, vexok_expr) || match(opcode, vexok_opcode_expr)) ++ flags = add_flags(flags, "INAT_VEXOK") ++ ++ # check prefixes ++ if (match(ext, prefix_expr)) { ++ if (!prefix_num[opcode]) ++ semantic_error("Unknown prefix: " opcode) ++ flags = add_flags(flags, "INAT_MAKE_PREFIX(" prefix_num[opcode] ")") ++ } ++ if (length(flags) == 0) ++ continue ++ # check if last prefix ++ if (match(ext, lprefix1_expr)) { ++ lptable1[idx] = add_flags(lptable1[idx],flags) ++ variant = "INAT_VARIANT" ++ } ++ if (match(ext, lprefix2_expr)) { ++ lptable2[idx] = add_flags(lptable2[idx],flags) ++ variant = "INAT_VARIANT" ++ } ++ if (match(ext, lprefix3_expr)) { ++ lptable3[idx] = add_flags(lptable3[idx],flags) ++ variant = "INAT_VARIANT" ++ } ++ if (!match(ext, lprefix_expr)){ ++ table[idx] = add_flags(table[idx],flags) ++ } ++ } ++ if (variant) ++ table[idx] = add_flags(table[idx],variant) ++} ++ ++END { ++ if (awkchecked != "") ++ exit 1 ++ # print escape opcode map's array ++ print "/* Escape opcode map array */" ++ print "const insn_attr_t * const inat_escape_tables[INAT_ESC_MAX + 1]" \ ++ "[INAT_LSTPFX_MAX + 1] = {" ++ for (i = 0; i < geid; i++) ++ for (j = 0; j < max_lprefix; j++) ++ if (etable[i,j]) ++ print " ["i"]["j"] = "etable[i,j]"," ++ print "};\n" ++ # print group opcode map's array ++ print "/* Group opcode map array */" ++ print "const insn_attr_t * const inat_group_tables[INAT_GRP_MAX + 1]"\ ++ "[INAT_LSTPFX_MAX + 1] = {" ++ for (i = 0; i < ggid; i++) ++ for (j = 0; j < max_lprefix; j++) ++ if (gtable[i,j]) ++ print " ["i"]["j"] = "gtable[i,j]"," ++ print "};\n" ++ # print AVX opcode map's array ++ print "/* AVX opcode map array */" ++ print "const insn_attr_t * const inat_avx_tables[X86_VEX_M_MAX + 1]"\ ++ "[INAT_LSTPFX_MAX + 1] = {" ++ for (i = 0; i < gaid; i++) ++ for (j = 0; j < max_lprefix; j++) ++ if (atable[i,j]) ++ print " ["i"]["j"] = "atable[i,j]"," ++ print "};" ++} ++ +diff --git a/tools/objtool/orc.h b/tools/objtool/orc.h +index a4139e386ef3..b0e92a6d0903 100644 +--- a/tools/objtool/orc.h ++++ b/tools/objtool/orc.h +@@ -18,7 +18,7 @@ + #ifndef _ORC_H + #define _ORC_H + +-#include "orc_types.h" ++#include + + struct objtool_file; + +diff --git a/tools/objtool/orc_dump.c b/tools/objtool/orc_dump.c +index 36c5bf6a2675..c3343820916a 100644 +--- a/tools/objtool/orc_dump.c ++++ b/tools/objtool/orc_dump.c +@@ -76,7 +76,8 @@ int orc_dump(const char *_objname) + int fd, nr_entries, i, *orc_ip = NULL, orc_size = 0; + struct orc_entry *orc = NULL; + char *name; +- unsigned long nr_sections, orc_ip_addr = 0; ++ size_t nr_sections; ++ Elf64_Addr orc_ip_addr = 0; + size_t shstrtab_idx; + Elf *elf; + Elf_Scn *scn; +@@ -187,10 +188,10 @@ int orc_dump(const char *_objname) + return -1; + } + +- printf("%s+%lx:", name, rela.r_addend); ++ printf("%s+%llx:", name, (unsigned long long)rela.r_addend); + + } else { +- printf("%lx:", orc_ip_addr + (i * sizeof(int)) + orc_ip[i]); ++ printf("%llx:", (unsigned long long)(orc_ip_addr + (i * sizeof(int)) + orc_ip[i])); + } + + +diff --git a/tools/objtool/orc_types.h b/tools/objtool/orc_types.h +deleted file mode 100644 +index 9c9dc579bd7d..000000000000 +--- a/tools/objtool/orc_types.h ++++ /dev/null +@@ -1,107 +0,0 @@ +-/* +- * Copyright (C) 2017 Josh Poimboeuf +- * +- * This program is free software; you can redistribute it and/or +- * modify it under the terms of the GNU General Public License +- * as published by the Free Software Foundation; either version 2 +- * of the License, or (at your option) any later version. +- * +- * This program is distributed in the hope that it will be useful, +- * but WITHOUT ANY WARRANTY; without even the implied warranty of +- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +- * GNU General Public License for more details. +- * +- * You should have received a copy of the GNU General Public License +- * along with this program; if not, see . +- */ +- +-#ifndef _ORC_TYPES_H +-#define _ORC_TYPES_H +- +-#include +-#include +- +-/* +- * The ORC_REG_* registers are base registers which are used to find other +- * registers on the stack. +- * +- * ORC_REG_PREV_SP, also known as DWARF Call Frame Address (CFA), is the +- * address of the previous frame: the caller's SP before it called the current +- * function. +- * +- * ORC_REG_UNDEFINED means the corresponding register's value didn't change in +- * the current frame. +- * +- * The most commonly used base registers are SP and BP -- which the previous SP +- * is usually based on -- and PREV_SP and UNDEFINED -- which the previous BP is +- * usually based on. +- * +- * The rest of the base registers are needed for special cases like entry code +- * and GCC realigned stacks. +- */ +-#define ORC_REG_UNDEFINED 0 +-#define ORC_REG_PREV_SP 1 +-#define ORC_REG_DX 2 +-#define ORC_REG_DI 3 +-#define ORC_REG_BP 4 +-#define ORC_REG_SP 5 +-#define ORC_REG_R10 6 +-#define ORC_REG_R13 7 +-#define ORC_REG_BP_INDIRECT 8 +-#define ORC_REG_SP_INDIRECT 9 +-#define ORC_REG_MAX 15 +- +-/* +- * ORC_TYPE_CALL: Indicates that sp_reg+sp_offset resolves to PREV_SP (the +- * caller's SP right before it made the call). Used for all callable +- * functions, i.e. all C code and all callable asm functions. +- * +- * ORC_TYPE_REGS: Used in entry code to indicate that sp_reg+sp_offset points +- * to a fully populated pt_regs from a syscall, interrupt, or exception. +- * +- * ORC_TYPE_REGS_IRET: Used in entry code to indicate that sp_reg+sp_offset +- * points to the iret return frame. +- * +- * The UNWIND_HINT macros are used only for the unwind_hint struct. They +- * aren't used in struct orc_entry due to size and complexity constraints. +- * Objtool converts them to real types when it converts the hints to orc +- * entries. +- */ +-#define ORC_TYPE_CALL 0 +-#define ORC_TYPE_REGS 1 +-#define ORC_TYPE_REGS_IRET 2 +-#define UNWIND_HINT_TYPE_SAVE 3 +-#define UNWIND_HINT_TYPE_RESTORE 4 +- +-#ifndef __ASSEMBLY__ +-/* +- * This struct is more or less a vastly simplified version of the DWARF Call +- * Frame Information standard. It contains only the necessary parts of DWARF +- * CFI, simplified for ease of access by the in-kernel unwinder. It tells the +- * unwinder how to find the previous SP and BP (and sometimes entry regs) on +- * the stack for a given code address. Each instance of the struct corresponds +- * to one or more code locations. +- */ +-struct orc_entry { +- s16 sp_offset; +- s16 bp_offset; +- unsigned sp_reg:4; +- unsigned bp_reg:4; +- unsigned type:2; +-} __packed; +- +-/* +- * This struct is used by asm and inline asm code to manually annotate the +- * location of registers on the stack for the ORC unwinder. +- * +- * Type can be either ORC_TYPE_* or UNWIND_HINT_TYPE_*. +- */ +-struct unwind_hint { +- u32 ip; +- s16 sp_offset; +- u8 sp_reg; +- u8 type; +-}; +-#endif /* __ASSEMBLY__ */ +- +-#endif /* _ORC_TYPES_H */ +diff --git a/tools/objtool/sync-check.sh b/tools/objtool/sync-check.sh +new file mode 100755 +index 000000000000..1470e74e9d66 +--- /dev/null ++++ b/tools/objtool/sync-check.sh +@@ -0,0 +1,29 @@ ++#!/bin/sh ++# SPDX-License-Identifier: GPL-2.0 ++ ++FILES=' ++arch/x86/lib/insn.c ++arch/x86/lib/inat.c ++arch/x86/lib/x86-opcode-map.txt ++arch/x86/tools/gen-insn-attr-x86.awk ++arch/x86/include/asm/insn.h ++arch/x86/include/asm/inat.h ++arch/x86/include/asm/inat_types.h ++arch/x86/include/asm/orc_types.h ++' ++ ++check() ++{ ++ local file=$1 ++ ++ diff $file ../../$file > /dev/null || ++ echo "Warning: synced file at 'tools/objtool/$file' differs from latest kernel version at '$file'" ++} ++ ++if [ ! -d ../../kernel ] || [ ! -d ../../tools ] || [ ! -d ../objtool ]; then ++ exit 0 ++fi ++ ++for i in $FILES; do ++ check $i ++done +diff --git a/tools/perf/util/intel-pt-decoder/x86-opcode-map.txt b/tools/perf/util/intel-pt-decoder/x86-opcode-map.txt +index 12e377184ee4..e0b85930dd77 100644 +--- a/tools/perf/util/intel-pt-decoder/x86-opcode-map.txt ++++ b/tools/perf/util/intel-pt-decoder/x86-opcode-map.txt +@@ -607,7 +607,7 @@ fb: psubq Pq,Qq | vpsubq Vx,Hx,Wx (66),(v1) + fc: paddb Pq,Qq | vpaddb Vx,Hx,Wx (66),(v1) + fd: paddw Pq,Qq | vpaddw Vx,Hx,Wx (66),(v1) + fe: paddd Pq,Qq | vpaddd Vx,Hx,Wx (66),(v1) +-ff: ++ff: UD0 + EndTable + + Table: 3-byte opcode 1 (0x0f 0x38) +@@ -717,7 +717,7 @@ AVXcode: 2 + 7e: vpermt2d/q Vx,Hx,Wx (66),(ev) + 7f: vpermt2ps/d Vx,Hx,Wx (66),(ev) + 80: INVEPT Gy,Mdq (66) +-81: INVPID Gy,Mdq (66) ++81: INVVPID Gy,Mdq (66) + 82: INVPCID Gy,Mdq (66) + 83: vpmultishiftqb Vx,Hx,Wx (66),(ev) + 88: vexpandps/d Vpd,Wpd (66),(ev) +@@ -896,7 +896,7 @@ EndTable + + GrpTable: Grp3_1 + 0: TEST Eb,Ib +-1: ++1: TEST Eb,Ib + 2: NOT Eb + 3: NEG Eb + 4: MUL AL,Eb +@@ -970,6 +970,15 @@ GrpTable: Grp9 + EndTable + + GrpTable: Grp10 ++# all are UD1 ++0: UD1 ++1: UD1 ++2: UD1 ++3: UD1 ++4: UD1 ++5: UD1 ++6: UD1 ++7: UD1 + EndTable + + # Grp11A and Grp11B are expressed as Grp11 in Intel SDM +diff --git a/tools/testing/selftests/x86/ldt_gdt.c b/tools/testing/selftests/x86/ldt_gdt.c +index 66e5ce5b91f0..0304ffb714f2 100644 +--- a/tools/testing/selftests/x86/ldt_gdt.c ++++ b/tools/testing/selftests/x86/ldt_gdt.c +@@ -627,13 +627,10 @@ static void do_multicpu_tests(void) + static int finish_exec_test(void) + { + /* +- * In a sensible world, this would be check_invalid_segment(0, 1); +- * For better or for worse, though, the LDT is inherited across exec. +- * We can probably change this safely, but for now we test it. ++ * Older kernel versions did inherit the LDT on exec() which is ++ * wrong because exec() starts from a clean state. + */ +- check_valid_segment(0, 1, +- AR_DPL3 | AR_TYPE_XRCODE | AR_S | AR_P | AR_DB, +- 42, true); ++ check_invalid_segment(0, 1); + + return nerrs ? 1 : 0; + } +diff --git a/virt/kvm/arm/mmu.c b/virt/kvm/arm/mmu.c +index b36945d49986..b4b69c2d1012 100644 +--- a/virt/kvm/arm/mmu.c ++++ b/virt/kvm/arm/mmu.c +@@ -509,8 +509,6 @@ static void unmap_hyp_range(pgd_t *pgdp, phys_addr_t start, u64 size) + */ + void free_hyp_pgds(void) + { +- unsigned long addr; +- + mutex_lock(&kvm_hyp_pgd_mutex); + + if (boot_hyp_pgd) { +@@ -521,10 +519,10 @@ void free_hyp_pgds(void) + + if (hyp_pgd) { + unmap_hyp_range(hyp_pgd, hyp_idmap_start, PAGE_SIZE); +- for (addr = PAGE_OFFSET; virt_addr_valid(addr); addr += PGDIR_SIZE) +- unmap_hyp_range(hyp_pgd, kern_hyp_va(addr), PGDIR_SIZE); +- for (addr = VMALLOC_START; is_vmalloc_addr((void*)addr); addr += PGDIR_SIZE) +- unmap_hyp_range(hyp_pgd, kern_hyp_va(addr), PGDIR_SIZE); ++ unmap_hyp_range(hyp_pgd, kern_hyp_va(PAGE_OFFSET), ++ (uintptr_t)high_memory - PAGE_OFFSET); ++ unmap_hyp_range(hyp_pgd, kern_hyp_va(VMALLOC_START), ++ VMALLOC_END - VMALLOC_START); + + free_pages((unsigned long)hyp_pgd, hyp_pgd_order); + hyp_pgd = NULL;