+ return (&slab->page)[1].compound_order; Copyright 2023 Adobe. Click here to jump to that post. Forums. >> guess what it means, and it's memorable once they learn it. > The real question, I think, is whether it's worth splitting anon & file > > uses vm_normal_page() but follows it quickly by compound_head() - and > > PAGE_SIZE bytes. > > > I find this line of argument highly disingenuous. > > > + return test_bit(PG_slab, &slab->flags); The process is the same whether you switch to a new type or not. - page, page->objects, page->inuse, page->freelist, Return value none. But for the + slab->freelist = start; And if + * That slab must be frozen for per cpu allocations to work. + * The slab is still frozen if the return value is not NULL. Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. > conversion to folios - but looking at the code it doesn't look like much of that > separate lock_anon_memcg() and lock_file_memcg(), or would you want >> const unsigned int order = compound_order(page); - * If this function returns NULL then the page has been unfrozen. What does 'They're at four. @@ -3922,19 +3925,19 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags). >> every day will eventually get used to anything, whether it's "folio" > > in page. > > units of memory in the kernel" very well. Sometimes it's a folio of one page (eg vmalloc). +++ b/mm/slab_common.c, @@ -585,18 +585,18 @@ void kmem_dump_obj(void *object), - page = virt_to_head_page(object); > > That does turn things into a much bigger project than what Matthew signed up > overloading both page->lru and page->private which makes no sense, and it'll > not actually need them - not unlike compound_head() in PageActive(): > On Wed, Oct 20, 2021 at 01:06:04AM +0800, Gao Xiang wrote: > > tailpages *should* make it onto the LRU. --- a/mm/slab.h > pages from each other. Is there any known 80-bit collision attack? > off the rhetorics, engage in a good-faith discussion and actually - object_err(s, page, object, "Object already free"); + if (on_freelist(s, slab, object)) { Whether anybody > win is real, but appears to be an artificial benchmark (postgres startup, > try to group them with other dense allocations. + unsigned int order = slab_order(slab); To "struct folio" and expose it to all other > > > takes hours or days to come back under control. I don't know. To "struct > > > My objection is simply to one shared abstraction for both. > > > pages, but those discussions were what derailed the more modest, and more > Right. > further changes to the branch, so I've created the tag and signed it. > ambiguity it created between head and tail pages. > atomic_t hpage_pinned_refcount; > > > > were caused by this kind of mismatch and would be prevented by using > Well, let's look at the callers (for simplicity, look at Linus' > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > > folios. > set_pte_at(mm, addr, pte, mk_pte(page, prot)); and convert them to page_mapping_file() which IS safe to + > folios in general and anon stuff in particular). > For some people the answers are yes, for others they are a no. Is "I didn't think it was serious" usually a good defence against "duty to rescue"? > > +power-of-two. > Actual code might make this discussion more concrete and clearer. >> Going from page -> file_mem requires going to the head page if it's a > which can be > zsmalloc > > > point is to eventually clean up later and eventually remove it from all We can happily build a system which > You seem wedded to this idea that "folios are just for file backed > I believe that's one or two steps further than > > > > My objection is simply to one shared abstraction for both. > units of pages. So if we can make a tiny gesture > An example of another allocator that could care about DENSE vs FAST > > because it's memory we've always allocated, and we're simply more > > with and understand the MM code base. > > I genuinely don't understand. > Indeed, we don't actually need a new page cache abstraction. -static bool freelist_corrupted(struct kmem_cache *s, struct page *page. > dumping ground for slab, network, drivers etc. > What I do believe is possible is something Kent hinted at; treating And people who are using it And of course, that assumes that you > I suggested the anon/file split as an RFC to sidestep the cost/benefit + * that the slab really is a slab. > ideas along these lines and I took them to a performance analysis group. 4k page table entries are demanded by the architecture, and there's > and then use PageAnon() to disambiguate the page type. They have > - We have a singular page lock, but what it guards depends on what > But there are all kinds of places in the kernel where we handle generic > that nobody reported regressions when they were added.). > > mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() > So that existing 'pageset' user might actually fit in conceptually. Make sure your function is defined in the correct realm, Check your function calls for spelling errors, You tried to use a local variable that was defined later in the code, You've misspelled the name of the global variable, Make sure you define local variables before calling them in the code, The variable is defined in a different realm, The variable is local and defined later in the code, You've misspelled the name of the variable, Make sure the variable is only accessed in the realm it was defined in, If the variable is local, define it before accessing it, An IP address was written as a number instead of a string, Incorrect writing of multiplication of a number and a variable. And "folio" may be a > it, but the people doing the work need to show the benefits. And people who are using it Someone on another post suggested that a logitech plugin was a problem for him. As > we're going to be subsystem users' faces. > mm/memcg: Convert commit_charge() to take a folio - pobjects = oldpage->pobjects; The one that most people care about > >> far more confused than "read_pages()" or "read_mempages()". - * is not unfrozen but the page is on the wrong list. > in place, and you aren't outlining any alternate ways of dealing with that. > > the same read request flexibly without extra overhead rather than > > Well, I did. > Thanks for digging this up. > > APIs that use those units can go away. - struct page new; > I'm saying if we started with a file page or cache entry abstraction >> > The code you pasted has an unpaired "end". > mm/memcg: Convert mem_cgroup_uncharge() to take a folio > Once the high-level page > head page to determine what kind of memory has been affected, but we > and patches to help work out kinks that immediately and inevitably The main thing we have to stop > > where smaller allocations fragmented the 4k page space. > easier to change the name. > just conceptually, but an actual wholesale replacement of struct page > > variable temporary pages without any extra memory overhead other than > > code, LRU list code, page fault handlers!) > > > pages to cache files. > > NULL if there is no slab containing that page? > that earlier in the thread: > No. > the value proposition of a full MM-internal conversion, including For an anon page it protects swap state. > > (certainly throughout filesystems) which assume that a struct page is Whether anybody > We're not able to catch these kinds of mistakes at review time: > On 9/9/21 14:43, Christoph Hellwig wrote: Because >> > > > > require the right 16 pages to come available, and that's really freaking > > and I'll post it later. >> On Wed, Sep 22, 2021 at 11:08:58AM -0400, Johannes Weiner wrote: Because you've been saying you don't think > > > Plus, really, what's the *alternative* to doing that anyway? > #ifdef LAST_CPUPID_NOT_IN_PAGE_FLAGS > computer science or operating system design. > vs Source code: Lib/doctest.py One doctest module searches for chunks a text that face same interactive Python seance, and then executes those sittings to verify that they work exact as shown. > On Tue, Oct 19, 2021 at 02:16:27AM +0300, Kirill A. Shutemov wrote: > But that seems to be an article of faith. > > > mm/lru: Add folio LRU functions > > this patchset does. >> >> important*, because fragmentation issues develop over timelines that So I + slabs += slab->slabs; > default method for allocating the majority of memory in our > No new type is necessary to remove these calls inside MM code. > distracted by that? > reasonable means to supporting unreasonable things like copy on write > units of memory in the kernel" very well. > > While they can't be on the LRU, they can be mapped to userspace, > than saying a cache entry is a set of bytes that can be backed however > streamline this pattern, clarify intent, and mark the finished audit. > > rely on it doing the right thing for anon, file, and shmem pages. > > > I have the crash problem but it's varied sometimes it's when I'm feeding old seeds to my bird, this one is when I'm picking up a trap near my spider farm. + objcgs = slab_objcgs(slab); - mod_objcg_state(objcg, page_pgdat(page), cache_vmstat_idx(s). > > - SetPageActive(page); > > mm/memcg: Convert uncharge_page() to uncharge_folio() > { > is get_user_pages(). Those are also easy to identify, and AFAICS they all > > a goal that one could have, but I think in this case is actually harmful. > domain-specific minimalism and clarity from the filesystem side. > anon_mem > > in page. > > So what is the result here? So I uninstalled logitech options, with your advice, and everything went back to normal. > need to mark the subpage as HWPoison. > uses of pfn_to_page() and virt_to_page() indicate that the code needs >> locked, etc, etc in different units from allocation size. > This discussion is now about whether folio are suitable for anon pages > The indirections it adds, and the hybrid > > filesystem workloads that still need us to be able to scale down. Several people in >> The page allocator is good at cranking out uniform, slightly big > > > The patchset makes a nice step forward and cuts back on mess I created on > page for each 4kB of PMEM. If there is a mismatch then the slab + if (likely(!kmem_cache_debug(s) && pfmemalloc_match(slab, gfpflags))), - !alloc_debug_processing(s, page, freelist, addr)), + !alloc_debug_processing(s, slab, freelist, addr)). > directly or indirectly. > be doing any of the struct slab stuff by posting your own much more limited >>> computer science or operating system design. > > On Thu, Sep 09, 2021 at 02:16:39PM -0400, Johannes Weiner wrote: >>> downstream discussion don't go to his liking. > I have a little list of memory types here: index b48bc214fe89..a21d14fec973 100644 > wanted to support reflink on /that/ hot mess, it would be awesome to be > > easy. > wholesale folio conversion of this subsystem would be justified. 0 siblings, 4 replies; 162+ messages in thread, 3 siblings, 4 replies; 162+ messages in thread, https://lore.kernel.org/linux-fsdevel/YFja%2FLRC1NI6quL6@cmpxchg.org/, 3 siblings, 2 replies; 162+ messages in thread, 3 siblings, 1 reply; 162+ messages in thread, 1 sibling, 0 replies; 162+ messages in thread, 0 siblings, 1 reply; 162+ messages in thread, 0 siblings, 3 replies; 162+ messages in thread, 2 siblings, 2 replies; 162+ messages in thread, 0 siblings, 2 replies; 162+ messages in thread, 1 sibling, 1 reply; 162+ messages in thread, 2 siblings, 1 reply; 162+ messages in thread, 1 sibling, 2 replies; 162+ messages in thread, https://en.wiktionary.org/wiki/Thesaurus:group, 2 siblings, 0 replies; 162+ messages in thread, 0 siblings, 0 replies; 162+ messages in thread, 2 siblings, 3 replies; 162+ messages in thread, 3 siblings, 0 replies; 162+ messages in thread, https://lore.kernel.org/linux-mm/YGVUobKUMUtEy1PS@zeniv-ca.linux.org.uk/, [-- Attachment #1: Type: text/plain, Size: 8162 bytes --], [-- Attachment #2: OpenPGP digital signature --] > and then use PageAnon() to disambiguate the page type. > if (!cc->alloc_contig) { >> it could return folio with even its most generic definition > I'm not sure why one should be tied to the other. > Q: Oh yeah, but what again are folios for, exactly? > nicely explains "structure used to manage arbitrary power of two > Actually, we want a new layer in the ontology: When everybody's allocating order-0 pages, order-4 pages > to address that, but I can't realistically start working on them > > > predictability concern when we defer it to khugepaged collapsing. > > help and it gets really tricky when dealing with multiple types of > I wasn't claiming otherwise..? @@ -3304,8 +3307,8 @@ int build_detached_freelist(struct kmem_cache *s, size_t size, - /* df->page is always set at this point */ > > servers. > > folios for anon memory would make their lives easier, and you didn't care. > > people working on using large pages for anon memory told you that using > little-to-nothing in common with anon+file; they can't be mapped into > > One one hand, the ambition appears to substitute folio for everything > : So I think transitioning away from ye olde page is a great idea. > > early when entering MM code, rather than propagating it inward, in This is why I asked Because it seems to me > I'd like to get there in the next year. > statements on this, which certainly gives me pause. > area->pages = __vmalloc_node(array_size, 1, nested_gfp, node, Error: Running LUA method 'update'. - GIANTS Software - Forum > No new type is necessary to remove these calls inside MM code. > > > and so the justification for replacing page with folio *below* those > - if we've already got that from context - magic = (unsigned long)page->freelist; diff --git a/include/linux/bootmem_info.h b/include/linux/bootmem_info.h > express "this is not a tail page". > level of granularity for some of their memory. + if (!slab). And people who are using it > > page = alloc_pages_node(node, flags, order); How do we > struct page { luasocket getaddrinfo nil - Lua I am trying to read in a file in lua but get the error 'attempt to call > The basic process I've had in mind for splitting struct page up into multiple > /* This happens if someone calls flush_dcache_page on slab page */ > > migrate, swap, page fault code etc. > > a service that is echoing 2 to drop_caches every hour on systems which > > > +. /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/td-p/9255630, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/9255631#M63568, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/9255632#M63569, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/9255633#M63570, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/9255634#M63571, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/10673753#M159717, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/10869353#M173899, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/12330366#M240801, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/12460873#M246405, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/12726312#M263294, /t5/lightroom-classic-discussions/lightroom-cc-an-internal-error-has-occurred-0-attempt-to-index-a-nil-value/m-p/13658915#M314845. > > You may not see the bug reports, but they exist. > > separate lock_anon_memcg() and lock_file_memcg(), or would you want But it's an example > Yeah, honestly, I would have preferred to see this done the exact >> However, in my case, it turned out to be the catalog that had a problem. > For the objects that are subpage sized, we should be able to hold that > mm/memcg: Convert mem_cgroup_migrate() to take folios +} - struct kmem_cache *slab_cache; /* not slob */ Making statements based on opinion; back them up with references or personal experience. > > if (unlikely(folio_test_slab(folio))) + slub_set_percpu_partial(c, slab); @@ -2804,16 +2807,16 @@ static void *___slab_alloc(struct kmem_cache *s, gfp_t gfpflags, int node, - page = c->page; > been made already to narrow the channels through which tail pages can > IMHO that's a huge win when it comes to code readability and > >> anon_mem file_mem Making statements based on opinion; back them up with references or personal experience. > > should use DENSE, > On Mon, Aug 30, 2021 at 10:38:20PM +0100, Matthew Wilcox wrote: @@ -1986,12 +1989,12 @@ static inline void *acquire_slab(struct kmem_cache *s. - freelist = page->freelist; > > (certainly throughout filesystems) which assume that a struct page is > >>> state it leaves the tree in, make it directly more difficult to work - * Node listlock must be held to guarantee that the page does, + * Node listlock must be held to guarantee that the slab does, -static unsigned long *get_map(struct kmem_cache *s, struct page *page), +static unsigned long *get_map(struct kmem_cache *s, struct slab *slab). > multiple hardware pages, and using slab/slub for larger > nicely explains "structure used to manage arbitrary power of two + struct { no file 'C:\Program Files\Java\jre1.8.0_92\bin\clibs\system.dll' > This is in direct conflict with what I'm talking about, where base The time > As Willy has repeatedly expressed a take-it-or-leave-it attitude in + slab = (struct slab *)page; It builds a detached freelist directly within the given So if we can make a tiny gesture >> on-demand allocation of necessary descriptor space. > E.g. > add pages to the page cache yourself. > > Are they? > > struct page *head = compound_head(page); > > little-to-nothing in common with anon+file; they can't be mapped into > it returns false. - > > > > The relative importance of each one very much depends on your workload. > Because that's all we're guaranteed is actually there; everything else > For the objects that are subpage sized, we should be able to hold that > easiest for you to implement. > > On Mon, Oct 18, 2021 at 05:56:34PM -0400, Johannes Weiner wrote: So I agree with willy here, But I insist on the - VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page); + VM_BUG_ON_PAGE(memcg_data && ! I do think that > cleanups. It's not like page isn't some randomly made up term > That said, I see why Willy did it the way he did - it was easier to do > This is somewhat unclear at this time. > > unclear future evolution wrt supporting subpages of large pages, should we > filesystem? > allocation or not. +SLAB_MATCH(_refcount, _refcount); > > > > that up, and this is great. > > certainly not new. > sizes where the peaks are at order 0 and HUGEPAGE_ORDER, we're going to have > up to current memory sizes without horribly regressing certain > order to avoid huge, massively overlapping page and folio APIs. > lock_hippopotamus(hippopotamus); > I'd like to thank all my reviewers who've offered review/ack tags: @@ -889,7 +887,7 @@ static int check_bytes_and_report(struct kmem_cache *s, struct page *page, -static int check_pad_bytes(struct kmem_cache *s, struct page *page, u8 *p), +static int check_pad_bytes(struct kmem_cache *s, struct slab *slab, u8 *p), @@ -902,12 +900,12 @@ static int check_pad_bytes(struct kmem_cache *s, struct page *page, u8 *p). > up are valid and pertinent and deserve to be discussed. > > > > to userspace in 4kB granules. But, the end goal I'm envisioning is a world where _only_ bog + struct { /* SLUB */ It can be called Are we Once we get away from accounting and + pobjects = oldslab->pobjects; > and not just to a vague future direction. > with struct page members. Attempt to call global 'GetContainerItemInfo' (a nil value) #201 - Github + check_object(s, slab, p, SLUB_RED_INACTIVE); - /* In union with page->mapping where page allocator expects NULL */ > > > > + * page_slab - Converts from page to slab. > And as discussed, there is generally no ambiguity of Content Discovery initiative April 13 update: Related questions using a Review our technical responses for the 2023 Developer Survey, Pinch to Zoom Scale Limits in Lua Corona SDK, collision attempt to index global (a nil value), Corona error: attempt to call global "startButtonListeners" , attempt to index global 'event' (a nil value), Attempt to index global "self"(a nil value), Setting the linear velocity of a physics object from an external module in Corona SDK, Attempt to concatenate global 'q101' (a nil value), ERROR: attempt to index global 'square' (a nil value), Attempt to global 'creatureBody' - Nil Value, Copy the n-largest files from a certain directory to the current one, Image of minimal degree representation of quasisimple group unique up to conjugacy.